Max level shown: TRCE DEBG INFO WARN ERRO FATA

1	2024-12-06T21:15:03.353Z	DEBG	nexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
2	2024-12-06T21:15:03.353Z	DEBG	nexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
3	2024-12-06T21:15:03.355Z	DEBG	nexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
4	2024-12-06T21:15:03.355Z	DEBG	nexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
5	2024-12-06T21:15:03.357Z	DEBG	nexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6	2024-12-06T21:15:03.359Z	DEBG	nexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: 2d75e4a3-6363-4fd1-935e-989d31fd7d71, lookup_type: ById(2d75e4a3-6363-4fd1-935e-989d31fd7d71) } result = Ok(())
7	2024-12-06T21:15:03.359Z	INFO	nexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: 2d75e4a3-6363-4fd1-935e-989d31fd7d71, time_created: 2024-12-06T21:14:54.225042Z, time_modified: 2024-12-06T21:14:54.225042Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:768
8	2024-12-06T21:15:04.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 2 reason = Timeout
9	2024-12-06T21:15:04.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 2 reason = Timeout
10	2024-12-06T21:15:04.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
11	2024-12-06T21:15:04.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12	2024-12-06T21:15:04.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13	2024-12-06T21:15:04.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
14	2024-12-06T21:15:04.408Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15	2024-12-06T21:15:04.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
16	2024-12-06T21:15:04.410Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17	2024-12-06T21:15:04.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18	2024-12-06T21:15:04.412Z	WARN	nexus (BackgroundTasks): failed to read target blueprint background_task = blueprint_loader error = Internal Error: no target blueprint set file = nexus/src/app/background/tasks/blueprint_load.rs:69
19	2024-12-06T21:15:04.412Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.561017ms iteration = 2 status = {"error":"failed to read target blueprint: Internal Error: no target blueprint set"}
20	2024-12-06T21:15:04.434Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
21	2024-12-06T21:15:04.434Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
22	2024-12-06T21:15:04.473Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 67.565988ms iteration = 2 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
23	2024-12-06T21:15:05.361Z	DEBG	nexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
24	2024-12-06T21:15:05.361Z	DEBG	nexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
25	2024-12-06T21:15:05.362Z	DEBG	nexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
26	2024-12-06T21:15:05.362Z	DEBG	nexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
27	2024-12-06T21:15:05.364Z	DEBG	nexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28	2024-12-06T21:15:05.367Z	DEBG	nexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: 2d75e4a3-6363-4fd1-935e-989d31fd7d71, lookup_type: ById(2d75e4a3-6363-4fd1-935e-989d31fd7d71) } result = Ok(())
29	2024-12-06T21:15:05.367Z	INFO	nexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: 2d75e4a3-6363-4fd1-935e-989d31fd7d71, time_created: 2024-12-06T21:14:54.225042Z, time_modified: 2024-12-06T21:14:54.225042Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:768
30	2024-12-06T21:15:07.368Z	DEBG	nexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
31	2024-12-06T21:15:07.368Z	DEBG	nexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
32	2024-12-06T21:15:07.369Z	DEBG	nexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
33	2024-12-06T21:15:07.369Z	DEBG	nexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
34	2024-12-06T21:15:07.372Z	DEBG	nexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35	2024-12-06T21:15:07.374Z	DEBG	nexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: 2d75e4a3-6363-4fd1-935e-989d31fd7d71, lookup_type: ById(2d75e4a3-6363-4fd1-935e-989d31fd7d71) } result = Ok(())
36	2024-12-06T21:15:07.374Z	INFO	nexus: Rack initialized file = nexus/src/app/rack.rs:765
37	2024-12-06T21:15:07.374Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
38	2024-12-06T21:15:07.374Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
39	2024-12-06T21:15:07.376Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
40	2024-12-06T21:15:07.376Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
41	2024-12-06T21:15:07.378Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
42	2024-12-06T21:15:07.378Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
43	2024-12-06T21:15:07.381Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
44	2024-12-06T21:15:07.381Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
45	2024-12-06T21:15:07.382Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
46	2024-12-06T21:15:07.382Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
47	2024-12-06T21:15:07.384Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
48	2024-12-06T21:15:07.384Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
49	2024-12-06T21:15:07.385Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50	2024-12-06T21:15:07.388Z	DEBG	nexus (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
51	2024-12-06T21:15:07.388Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
52	2024-12-06T21:15:07.389Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
53	2024-12-06T21:15:07.390Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
54	2024-12-06T21:15:07.390Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
55	2024-12-06T21:15:07.391Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
56	2024-12-06T21:15:07.391Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
57	2024-12-06T21:15:07.392Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58	2024-12-06T21:15:07.395Z	DEBG	nexus (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
59	2024-12-06T21:15:07.395Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
60	2024-12-06T21:15:07.395Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
61	2024-12-06T21:15:07.398Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
62	2024-12-06T21:15:07.399Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
63	2024-12-06T21:15:07.400Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
64	2024-12-06T21:15:07.400Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
65	2024-12-06T21:15:07.401Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
66	2024-12-06T21:15:07.401Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
67	2024-12-06T21:15:07.402Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
68	2024-12-06T21:15:07.403Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
69	2024-12-06T21:15:07.404Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
70	2024-12-06T21:15:07.404Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
71	2024-12-06T21:15:07.405Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
72	2024-12-06T21:15:07.405Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
73	2024-12-06T21:15:07.406Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74	2024-12-06T21:15:07.410Z	DEBG	nexus (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
75	2024-12-06T21:15:07.410Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
76	2024-12-06T21:15:07.410Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
77	2024-12-06T21:15:07.411Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
78	2024-12-06T21:15:07.411Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
79	2024-12-06T21:15:07.412Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
80	2024-12-06T21:15:07.413Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
81	2024-12-06T21:15:07.417Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
82	2024-12-06T21:15:07.417Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
83	2024-12-06T21:15:07.420Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
84	2024-12-06T21:15:07.421Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
85	2024-12-06T21:15:07.422Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
86	2024-12-06T21:15:07.422Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
87	2024-12-06T21:15:07.423Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
88	2024-12-06T21:15:07.423Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
89	2024-12-06T21:15:07.425Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90	2024-12-06T21:15:07.428Z	DEBG	nexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
91	2024-12-06T21:15:07.428Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
92	2024-12-06T21:15:07.428Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
93	2024-12-06T21:15:07.429Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
94	2024-12-06T21:15:07.430Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
95	2024-12-06T21:15:07.431Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
96	2024-12-06T21:15:07.431Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
97	2024-12-06T21:15:07.432Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98	2024-12-06T21:15:07.435Z	DEBG	nexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
99	2024-12-06T21:15:07.435Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
100	2024-12-06T21:15:07.435Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
101	2024-12-06T21:15:07.440Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
102	2024-12-06T21:15:07.440Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
103	2024-12-06T21:15:07.442Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
104	2024-12-06T21:15:07.442Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
105	2024-12-06T21:15:07.443Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
106	2024-12-06T21:15:07.444Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
107	2024-12-06T21:15:07.446Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
108	2024-12-06T21:15:07.446Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
109	2024-12-06T21:15:07.448Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
110	2024-12-06T21:15:07.448Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
111	2024-12-06T21:15:07.449Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
112	2024-12-06T21:15:07.449Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
113	2024-12-06T21:15:07.450Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
114	2024-12-06T21:15:07.450Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
115	2024-12-06T21:15:07.451Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116	2024-12-06T21:15:07.454Z	DEBG	nexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
117	2024-12-06T21:15:07.454Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
118	2024-12-06T21:15:07.455Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
119	2024-12-06T21:15:07.456Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
120	2024-12-06T21:15:07.456Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
121	2024-12-06T21:15:07.457Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
122	2024-12-06T21:15:07.457Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
123	2024-12-06T21:15:07.458Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124	2024-12-06T21:15:07.461Z	DEBG	nexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
125	2024-12-06T21:15:07.462Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
126	2024-12-06T21:15:07.462Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
127	2024-12-06T21:15:07.470Z	DEBG	nexus (InternalApi): resolved names for firewall rules instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 112d04a8-d7b2-41ba-a8a9-b1b05b82789a, kind: Service { id: e8725c8c-13ab-4916-bf16-bb7f832014d5 }, name: Name("nexus-e8725c8c-13ab-4916-bf16-bb7f832014d5"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 245, 172])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 3b455a5c-72a6-4c48-bc31-0f3337db4efa, kind: Service { id: a211d4f7-7350-4466-97ca-36b58597b187 }, name: Name("nexus-a211d4f7-7350-4466-97ca-36b58597b187"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 181, 41])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 73f8344b-aadb-4eda-971a-b52bf9a56847, kind: Service { id: 0214b950-30f3-4a60-9344-b4f7d110e307 }, name: Name("nexus-0214b950-30f3-4a60-9344-b4f7d110e307"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 172, 92])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 470a6d71-26ee-4d59-b30a-7a9e78dfa903, kind: Service { id: 709da000-e0da-4b29-8160-3112fd080927 }, name: Name("external-dns-709da000-e0da-4b29-8160-3112fd080927"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 160, 53])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8dee8d46-edc9-4f8a-9453-2146d512c47b, kind: Service { id: a03854a6-7a78-402b-93b4-65c6d36bf236 }, name: Name("external-dns-a03854a6-7a78-402b-93b4-65c6d36bf236"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 200])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
128	2024-12-06T21:15:07.470Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
129	2024-12-06T21:15:07.470Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
130	2024-12-06T21:15:07.471Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131	2024-12-06T21:15:07.473Z	DEBG	nexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
132	2024-12-06T21:15:07.473Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
133	2024-12-06T21:15:07.473Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
134	2024-12-06T21:15:07.475Z	DEBG	nexus (InternalApi): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified.
135	2024-12-06T21:15:07.478Z	DEBG	nexus (InternalApi): resolved firewall rules for sled agents sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 470a6d71-26ee-4d59-b30a-7a9e78dfa903, kind: Service { id: 709da000-e0da-4b29-8160-3112fd080927 }, name: Name("external-dns-709da000-e0da-4b29-8160-3112fd080927"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 160, 53])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8dee8d46-edc9-4f8a-9453-2146d512c47b, kind: Service { id: a03854a6-7a78-402b-93b4-65c6d36bf236 }, name: Name("external-dns-a03854a6-7a78-402b-93b4-65c6d36bf236"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 200])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 112d04a8-d7b2-41ba-a8a9-b1b05b82789a, kind: Service { id: e8725c8c-13ab-4916-bf16-bb7f832014d5 }, name: Name("nexus-e8725c8c-13ab-4916-bf16-bb7f832014d5"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 245, 172])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 3b455a5c-72a6-4c48-bc31-0f3337db4efa, kind: Service { id: a211d4f7-7350-4466-97ca-36b58597b187 }, name: Name("nexus-a211d4f7-7350-4466-97ca-36b58597b187"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 181, 41])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 73f8344b-aadb-4eda-971a-b52bf9a56847, kind: Service { id: 0214b950-30f3-4a60-9344-b4f7d110e307 }, name: Name("nexus-0214b950-30f3-4a60-9344-b4f7d110e307"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 172, 92])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
136	2024-12-06T21:15:07.478Z	DEBG	nexus (InternalApi): resolved 2 rules for sleds
137	2024-12-06T21:15:07.612Z	DEBG	nexus (InternalApi): resolved sleds for vpc oxide-services vpc_to_sled = [Sled { identity: SledIdentity { id: cbc5c1c6-5668-455a-85ef-94f4aebe0f80, time_created: 2024-12-06T21:15:03.870849Z, time_modified: 2024-12-06T21:15:03.870849Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 2d75e4a3-6363-4fd1-935e-989d31fd7d71, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(51474595840)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)) }]
138	2024-12-06T21:15:07.612Z	DEBG	nexus (InternalApi): sending firewall rules to sled agents
139	2024-12-06T21:15:07.612Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
140	2024-12-06T21:15:07.612Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
141	2024-12-06T21:15:07.614Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {} }
142	2024-12-06T21:15:07.614Z	DEBG	nexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
143	2024-12-06T21:15:07.616Z	DEBG	nexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144	2024-12-06T21:15:07.618Z	DEBG	nexus (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: cbc5c1c6-5668-455a-85ef-94f4aebe0f80, lookup_type: ById(cbc5c1c6-5668-455a-85ef-94f4aebe0f80) } result = Ok(())
145	2024-12-06T21:15:07.652Z	DEBG	nexus (InternalApi): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
146	2024-12-06T21:15:07.653Z	DEBG	nexus (InternalApi): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "6d938493-c8cd-4c0f-9b9b-2ccd4ddfd1e7", "date": "Fri, 06 Dec 2024 21:15:07 GMT"} })
147	2024-12-06T21:15:07.653Z	INFO	nexus (InternalApi): updated firewall rules on 1 sleds file = nexus/networking/src/firewall_rules.rs:496
148	2024-12-06T21:15:07.653Z	INFO	nexus: plumbed initial IP allowlist file = nexus/src/app/allow_list.rs:161
149	2024-12-06T21:15:14.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 3 reason = Timeout
150	2024-12-06T21:15:14.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 3 reason = Timeout
151	2024-12-06T21:15:14.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
152	2024-12-06T21:15:14.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
153	2024-12-06T21:15:14.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
154	2024-12-06T21:15:14.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
155	2024-12-06T21:15:14.409Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156	2024-12-06T21:15:14.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
157	2024-12-06T21:15:14.411Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
158	2024-12-06T21:15:14.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
159	2024-12-06T21:15:14.413Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
160	2024-12-06T21:15:14.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
161	2024-12-06T21:15:14.418Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
162	2024-12-06T21:15:14.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
163	2024-12-06T21:15:14.421Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
164	2024-12-06T21:15:14.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
165	2024-12-06T21:15:14.436Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
166	2024-12-06T21:15:14.437Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
167	2024-12-06T21:15:14.461Z	INFO	nexus (BackgroundTasks): found new target blueprint (first find) background_task = blueprint_loader file = nexus/src/app/background/tasks/blueprint_load.rs:88 target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
168	2024-12-06T21:15:14.461Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 55.612792ms iteration = 3 status = {"enabled":false,"status":"first target blueprint","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z","time_found":"2024-12-06T21:15:14.461612237Z"}
169	2024-12-06T21:15:14.461Z	DEBG	nexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 2 reason = Dependency
170	2024-12-06T21:15:14.461Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 2 reason = Dependency
171	2024-12-06T21:15:14.461Z	WARN	nexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0
172	2024-12-06T21:15:14.461Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
173	2024-12-06T21:15:14.461Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 22.58\u{b5}s iteration = 2 status = {"enabled":false,"target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0"}
174	2024-12-06T21:15:14.461Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
175	2024-12-06T21:15:14.461Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
176	2024-12-06T21:15:14.462Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
177	2024-12-06T21:15:14.462Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
178	2024-12-06T21:15:14.462Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
179	2024-12-06T21:15:14.462Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
180	2024-12-06T21:15:14.462Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
181	2024-12-06T21:15:14.462Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
182	2024-12-06T21:15:14.462Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
183	2024-12-06T21:15:14.464Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184	2024-12-06T21:15:14.466Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
185	2024-12-06T21:15:14.466Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
186	2024-12-06T21:15:14.466Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
187	2024-12-06T21:15:14.466Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
188	2024-12-06T21:15:14.468Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
189	2024-12-06T21:15:14.468Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
190	2024-12-06T21:15:14.468Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
191	2024-12-06T21:15:14.468Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
192	2024-12-06T21:15:14.470Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
193	2024-12-06T21:15:14.470Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
194	2024-12-06T21:15:14.470Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
195	2024-12-06T21:15:14.470Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
196	2024-12-06T21:15:14.471Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
197	2024-12-06T21:15:14.472Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
198	2024-12-06T21:15:14.472Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
199	2024-12-06T21:15:14.472Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
200	2024-12-06T21:15:14.473Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 67.442207ms iteration = 3 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
201	2024-12-06T21:15:14.473Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
202	2024-12-06T21:15:14.473Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
203	2024-12-06T21:15:14.474Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
204	2024-12-06T21:15:14.506Z	DEBG	nexus (BackgroundTasks): client request background_task = crdb_node_id_collector body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
205	2024-12-06T21:15:14.540Z	DEBG	nexus (BackgroundTasks): client request background_task = crdb_node_id_collector body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
206	2024-12-06T21:15:14.572Z	DEBG	nexus (BackgroundTasks): client request background_task = crdb_node_id_collector body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
207	2024-12-06T21:15:14.604Z	DEBG	nexus (BackgroundTasks): client request background_task = crdb_node_id_collector body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
208	2024-12-06T21:15:14.605Z	DEBG	nexus (BackgroundTasks): client response background_task = crdb_node_id_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "517c71e6-1a09-4b67-b6b2-3688041caed5", "content-length": "64", "date": "Fri, 06 Dec 2024 21:15:14 GMT"} })
209	2024-12-06T21:15:14.607Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
210	2024-12-06T21:15:14.607Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
211	2024-12-06T21:15:14.607Z	DEBG	nexus (BackgroundTasks): client response background_task = crdb_node_id_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a91c2d60-56be-4a85-b1f7-d56edbcf86db", "content-length": "64", "date": "Fri, 06 Dec 2024 21:15:14 GMT"} })
212	2024-12-06T21:15:14.607Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
213	2024-12-06T21:15:14.607Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
214	2024-12-06T21:15:14.607Z	DEBG	nexus (BackgroundTasks): client response background_task = crdb_node_id_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "16d230b8-1d91-48b6-ae47-15be6724c885", "content-length": "64", "date": "Fri, 06 Dec 2024 21:15:14 GMT"} })
215	2024-12-06T21:15:14.607Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
216	2024-12-06T21:15:14.607Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
217	2024-12-06T21:15:14.609Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
218	2024-12-06T21:15:14.610Z	DEBG	nexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
219	2024-12-06T21:15:14.610Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
220	2024-12-06T21:15:14.610Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
221	2024-12-06T21:15:14.610Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
222	2024-12-06T21:15:14.611Z	DEBG	nexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
223	2024-12-06T21:15:14.611Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
224	2024-12-06T21:15:14.611Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
225	2024-12-06T21:15:14.611Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
226	2024-12-06T21:15:14.612Z	DEBG	nexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
227	2024-12-06T21:15:14.612Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
228	2024-12-06T21:15:14.612Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
229	2024-12-06T21:15:14.643Z	DEBG	nexus (BackgroundTasks): client response background_task = crdb_node_id_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9081c91b-b425-476c-9076-8658ff8f5231", "content-length": "64", "date": "Fri, 06 Dec 2024 21:15:14 GMT"} })
230	2024-12-06T21:15:14.643Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
231	2024-12-06T21:15:14.643Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
232	2024-12-06T21:15:14.644Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
233	2024-12-06T21:15:14.646Z	DEBG	nexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
234	2024-12-06T21:15:14.646Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
235	2024-12-06T21:15:14.646Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
236	2024-12-06T21:15:14.651Z	DEBG	nexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 189.931656ms iteration = 2 status = {"nsuccess":5}
237	2024-12-06T21:15:21.534Z	INFO	nexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:60694
238	2024-12-06T21:15:21.541Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = d183ec06-4c7e-44b5-bb81-e99b5ff49335 roles = RoleSet { roles: {} } uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
239	2024-12-06T21:15:21.542Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = d183ec06-4c7e-44b5-bb81-e99b5ff49335 resource = Database result = Ok(()) uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
240	2024-12-06T21:15:21.548Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 14397 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = d183ec06-4c7e-44b5-bb81-e99b5ff49335 response_code = 200 uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
241	2024-12-06T21:15:21.550Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = 169f7604-b59f-40a7-8063-e507dab59b08 roles = RoleSet { roles: {} } uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjYmM1YzFjNi01NjY4LTQ1NWEtODVlZi05NGY0YWViZTBmODAifX0%3D
242	2024-12-06T21:15:21.550Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = 169f7604-b59f-40a7-8063-e507dab59b08 resource = Database result = Ok(()) uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjYmM1YzFjNi01NjY4LTQ1NWEtODVlZi05NGY0YWViZTBmODAifX0%3D
243	2024-12-06T21:15:21.552Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3122 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = 169f7604-b59f-40a7-8063-e507dab59b08 response_code = 200 uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjYmM1YzFjNi01NjY4LTQ1NWEtODVlZi05NGY0YWViZTBmODAifX0%3D
244	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 2 reason = Timeout
245	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 2 reason = Timeout
246	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
247	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
248	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 2 reason = Timeout
249	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
250	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 2 reason = Timeout
251	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 4 reason = Timeout
252	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 2 reason = Timeout
253	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
254	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
255	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
256	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 4 reason = Timeout
257	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
258	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 2 reason = Timeout
259	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 2 reason = Timeout
260	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
261	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 2 reason = Timeout
262	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 2 reason = Timeout
263	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 3 reason = Timeout
264	2024-12-06T21:15:24.406Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
265	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
266	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
267	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
268	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 2 reason = Timeout
269	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
270	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
271	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 2 reason = Timeout
272	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
273	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 2 reason = Timeout
274	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
275	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 2 reason = Timeout
276	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
277	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 2 reason = Timeout
278	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
279	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
280	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
281	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
282	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
283	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
284	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
285	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
286	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
287	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
288	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
289	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
290	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
291	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
292	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
293	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
294	2024-12-06T21:15:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
295	2024-12-06T21:15:24.409Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
296	2024-12-06T21:15:24.409Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
297	2024-12-06T21:15:24.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
298	2024-12-06T21:15:24.410Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
299	2024-12-06T21:15:24.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
300	2024-12-06T21:15:24.411Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
301	2024-12-06T21:15:24.411Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
302	2024-12-06T21:15:24.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
303	2024-12-06T21:15:24.412Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 6.742061ms iteration = 2 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
304	2024-12-06T21:15:24.413Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 839, tv_nsec: 912179184 } })
305	2024-12-06T21:15:24.413Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
306	2024-12-06T21:15:24.414Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
307	2024-12-06T21:15:24.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
308	2024-12-06T21:15:24.414Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
309	2024-12-06T21:15:24.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
310	2024-12-06T21:15:24.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
311	2024-12-06T21:15:24.414Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
312	2024-12-06T21:15:24.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
313	2024-12-06T21:15:24.416Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
314	2024-12-06T21:15:24.416Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
315	2024-12-06T21:15:24.416Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
316	2024-12-06T21:15:24.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
317	2024-12-06T21:15:24.416Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
318	2024-12-06T21:15:24.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
319	2024-12-06T21:15:24.418Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
320	2024-12-06T21:15:24.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
321	2024-12-06T21:15:24.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
322	2024-12-06T21:15:24.419Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
323	2024-12-06T21:15:24.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
324	2024-12-06T21:15:24.422Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
325	2024-12-06T21:15:24.422Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
326	2024-12-06T21:15:24.422Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
327	2024-12-06T21:15:24.423Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 839, tv_nsec: 921887533 } })
328	2024-12-06T21:15:24.423Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
329	2024-12-06T21:15:24.423Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
330	2024-12-06T21:15:24.423Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
331	2024-12-06T21:15:24.424Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
332	2024-12-06T21:15:24.424Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
333	2024-12-06T21:15:24.424Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
334	2024-12-06T21:15:24.427Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
335	2024-12-06T21:15:24.427Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
336	2024-12-06T21:15:24.429Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
337	2024-12-06T21:15:24.430Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
338	2024-12-06T21:15:24.430Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
339	2024-12-06T21:15:24.431Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
340	2024-12-06T21:15:24.436Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 839, tv_nsec: 935588588 } })
341	2024-12-06T21:15:24.437Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
342	2024-12-06T21:15:24.438Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
343	2024-12-06T21:15:24.442Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
344	2024-12-06T21:15:24.442Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
345	2024-12-06T21:15:24.442Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
346	2024-12-06T21:15:24.451Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
347	2024-12-06T21:15:24.452Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
348	2024-12-06T21:15:24.452Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
349	2024-12-06T21:15:24.452Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
350	2024-12-06T21:15:24.453Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 47.690482ms iteration = 4 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
351	2024-12-06T21:15:24.454Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
352	2024-12-06T21:15:24.455Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
353	2024-12-06T21:15:24.456Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
354	2024-12-06T21:15:24.456Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
355	2024-12-06T21:15:24.457Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
356	2024-12-06T21:15:24.457Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
357	2024-12-06T21:15:24.457Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
358	2024-12-06T21:15:24.457Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
359	2024-12-06T21:15:24.457Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
360	2024-12-06T21:15:24.457Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
361	2024-12-06T21:15:24.458Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
362	2024-12-06T21:15:24.459Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 53.516408ms iteration = 2 status = {"errors":[],"garbage_collect_requested":[]}
363	2024-12-06T21:15:24.460Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 54.497854ms iteration = 2 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
364	2024-12-06T21:15:24.461Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cbc5c1c6-5668-455a-85ef-94f4aebe0f80, lookup_type: ById(cbc5c1c6-5668-455a-85ef-94f4aebe0f80) } result = Ok(())
365	2024-12-06T21:15:24.461Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
366	2024-12-06T21:15:24.461Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
367	2024-12-06T21:15:24.462Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
368	2024-12-06T21:15:24.462Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
369	2024-12-06T21:15:24.463Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
370	2024-12-06T21:15:24.463Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 57.171058ms iteration = 4 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
371	2024-12-06T21:15:24.463Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 57.160877ms iteration = 2 status = {"errors":[],"records_set_to_done":[]}
372	2024-12-06T21:15:24.463Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 57.444312ms iteration = 2 status = 0
373	2024-12-06T21:15:24.470Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
374	2024-12-06T21:15:24.470Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 64.162252ms iteration = 2 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
375	2024-12-06T21:15:24.489Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 83.667212ms iteration = 2 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
376	2024-12-06T21:15:24.494Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
377	2024-12-06T21:15:24.494Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
378	2024-12-06T21:15:24.494Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b9b15385-1faa-4f7a-963e-72ff532faaeb", "content-length": "26", "date": "Fri, 06 Dec 2024 21:15:24 GMT"} })
379	2024-12-06T21:15:24.494Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
380	2024-12-06T21:15:24.495Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
381	2024-12-06T21:15:24.496Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
382	2024-12-06T21:15:24.496Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
383	2024-12-06T21:15:24.496Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
384	2024-12-06T21:15:24.497Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
385	2024-12-06T21:15:24.498Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 92.347635ms iteration = 2 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
386	2024-12-06T21:15:24.516Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 zone_address = fd00:1122:3344:101::2
387	2024-12-06T21:15:24.516Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
388	2024-12-06T21:15:24.516Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "00fa8e4c-16d3-46b3-985a-e1cb98d7cc5f", "content-length": "26", "date": "Fri, 06 Dec 2024 21:15:24 GMT"} })
389	2024-12-06T21:15:24.516Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
390	2024-12-06T21:15:24.518Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
391	2024-12-06T21:15:24.530Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
392	2024-12-06T21:15:24.530Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
393	2024-12-06T21:15:24.530Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 124.643654ms iteration = 2 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
394	2024-12-06T21:15:24.539Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 133.542321ms iteration = 3 status = {"physical_disks_added":0}
395	2024-12-06T21:15:24.540Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
396	2024-12-06T21:15:24.541Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
397	2024-12-06T21:15:24.592Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
398	2024-12-06T21:15:24.592Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
399	2024-12-06T21:15:24.592Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e2b5dc69-ef26-4c73-bf16-d0acbd9e9b7b", "content-length": "26", "date": "Fri, 06 Dec 2024 21:15:24 GMT"} })
400	2024-12-06T21:15:24.592Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
401	2024-12-06T21:15:24.594Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
402	2024-12-06T21:15:24.634Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
403	2024-12-06T21:15:24.634Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
404	2024-12-06T21:15:24.634Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "9b33bedd-9aa2-4bc0-9d09-78171d4a7d67", "content-length": "372", "date": "Fri, 06 Dec 2024 21:15:24 GMT"} })
405	2024-12-06T21:15:24.634Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
406	2024-12-06T21:15:24.635Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
407	2024-12-06T21:15:24.658Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
408	2024-12-06T21:15:24.658Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
409	2024-12-06T21:15:24.658Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
410	2024-12-06T21:15:24.658Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "6e31ff92-6a0e-411d-86f7-ab0cc463b44d", "date": "Fri, 06 Dec 2024 21:15:24 GMT"} })
411	2024-12-06T21:15:24.658Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
412	2024-12-06T21:15:24.658Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
413	2024-12-06T21:15:24.658Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
414	2024-12-06T21:15:24.659Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
415	2024-12-06T21:15:24.659Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "b962bbf3-f6c2-435a-8033-314a84ae337f", "content-length": "1", "date": "Fri, 06 Dec 2024 21:15:24 GMT"} })
416	2024-12-06T21:15:24.660Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
417	2024-12-06T21:15:24.660Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
418	2024-12-06T21:15:24.660Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
419	2024-12-06T21:15:24.661Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
420	2024-12-06T21:15:24.662Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
421	2024-12-06T21:15:24.662Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
422	2024-12-06T21:15:24.662Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
423	2024-12-06T21:15:24.662Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
424	2024-12-06T21:15:24.663Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 257.070687ms iteration = 2 status = null
425	2024-12-06T21:15:24.663Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
426	2024-12-06T21:15:24.665Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
427	2024-12-06T21:15:24.665Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
428	2024-12-06T21:15:24.665Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
429	2024-12-06T21:15:24.668Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
430	2024-12-06T21:15:24.668Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
431	2024-12-06T21:15:24.675Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
432	2024-12-06T21:15:24.677Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
433	2024-12-06T21:15:24.677Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "a45f7a80-a3fa-496d-96b9-10de31f921e7", "date": "Fri, 06 Dec 2024 21:15:24 GMT"} })
434	2024-12-06T21:15:24.678Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 272.289256ms iteration = 2 status = {}
435	2024-12-06T21:15:24.780Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 374.609124ms iteration = 2 status = {}
436	2024-12-06T21:15:24.783Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
437	2024-12-06T21:15:24.783Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "595bc0fa-3457-4952-bb05-04275f6fc180", "content-length": "691", "date": "Fri, 06 Dec 2024 21:15:24 GMT"} })
438	2024-12-06T21:15:24.785Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
439	2024-12-06T21:15:24.785Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
440	2024-12-06T21:15:24.785Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
441	2024-12-06T21:15:24.787Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 381.214933ms iteration = 2 status = {}
442	2024-12-06T21:15:24.824Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
443	2024-12-06T21:15:24.825Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
444	2024-12-06T21:15:24.827Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 3d56b7bf-53ed-4147-b45b-9cef154d7338, rack_id: 2d75e4a3-6363-4fd1-935e-989d31fd7d71, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(02859c9b-92d7-4b19-b425-8f266763be54) } rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
445	2024-12-06T21:15:24.827Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
446	2024-12-06T21:15:24.827Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
447	2024-12-06T21:15:24.846Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
448	2024-12-06T21:15:24.846Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "73638dbc-584a-4374-8b79-eb98acb21ded", "content-length": "134", "date": "Fri, 06 Dec 2024 21:15:24 GMT"} })
449	2024-12-06T21:15:24.848Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
450	2024-12-06T21:15:24.848Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 02859c9b-92d7-4b19-b425-8f266763be54
451	2024-12-06T21:15:24.853Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
452	2024-12-06T21:15:24.853Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
453	2024-12-06T21:15:24.853Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
454	2024-12-06T21:15:24.853Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
455	2024-12-06T21:15:24.853Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
456	2024-12-06T21:15:24.853Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
457	2024-12-06T21:15:24.855Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
458	2024-12-06T21:15:24.855Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
459	2024-12-06T21:15:24.855Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
460	2024-12-06T21:15:24.855Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
461	2024-12-06T21:15:24.856Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
462	2024-12-06T21:15:24.858Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
463	2024-12-06T21:15:24.860Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "06a13c2a-d405-4983-8643-12ede70b2b42", "content-length": "2", "date": "Fri, 06 Dec 2024 21:15:24 GMT"} })
464	2024-12-06T21:15:24.860Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
465	2024-12-06T21:15:24.860Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "f27f863c-01a6-4d39-9f6f-f7d50d1f37a7", "content-length": "52", "date": "Fri, 06 Dec 2024 21:15:24 GMT"} })
466	2024-12-06T21:15:24.860Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
467	2024-12-06T21:15:24.860Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
468	2024-12-06T21:15:24.960Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "63ebbaf9-dac9-46d5-a881-284a52e8f141", "date": "Fri, 06 Dec 2024 21:15:24 GMT"} })
469	2024-12-06T21:15:24.962Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch = Switch0
470	2024-12-06T21:15:24.962Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
471	2024-12-06T21:15:24.962Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b37d938-a8bd-4d25-b1bb-36fe3d9dcfd2", "content-length": "682", "date": "Fri, 06 Dec 2024 21:15:24 GMT"} })
472	2024-12-06T21:15:24.962Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
473	2024-12-06T21:15:24.962Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
474	2024-12-06T21:15:24.966Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
475	2024-12-06T21:15:24.967Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
476	2024-12-06T21:15:24.970Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
477	2024-12-06T21:15:24.970Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
478	2024-12-06T21:15:24.972Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
479	2024-12-06T21:15:24.973Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
480	2024-12-06T21:15:24.983Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 577.405371ms iteration = 2 status = {}
481	2024-12-06T21:15:34.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 5 reason = Timeout
482	2024-12-06T21:15:34.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 5 reason = Timeout
483	2024-12-06T21:15:34.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
484	2024-12-06T21:15:34.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
485	2024-12-06T21:15:34.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
486	2024-12-06T21:15:34.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
487	2024-12-06T21:15:34.408Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
488	2024-12-06T21:15:34.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
489	2024-12-06T21:15:34.409Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
490	2024-12-06T21:15:34.411Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.227809ms iteration = 5 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
491	2024-12-06T21:15:34.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
492	2024-12-06T21:15:34.412Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
493	2024-12-06T21:15:34.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
494	2024-12-06T21:15:34.414Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
495	2024-12-06T21:15:34.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
496	2024-12-06T21:15:34.415Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
497	2024-12-06T21:15:34.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
498	2024-12-06T21:15:34.418Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
499	2024-12-06T21:15:34.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
500	2024-12-06T21:15:34.460Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
501	2024-12-06T21:15:34.460Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 54.029616ms iteration = 5 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
502	2024-12-06T21:15:44.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 6 reason = Timeout
503	2024-12-06T21:15:44.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 6 reason = Timeout
504	2024-12-06T21:15:44.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
505	2024-12-06T21:15:44.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
506	2024-12-06T21:15:44.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
507	2024-12-06T21:15:44.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
508	2024-12-06T21:15:44.408Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
509	2024-12-06T21:15:44.408Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
510	2024-12-06T21:15:44.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
511	2024-12-06T21:15:44.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
512	2024-12-06T21:15:44.411Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
513	2024-12-06T21:15:44.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
514	2024-12-06T21:15:44.412Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
515	2024-12-06T21:15:44.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
516	2024-12-06T21:15:44.414Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
517	2024-12-06T21:15:44.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
518	2024-12-06T21:15:44.417Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
519	2024-12-06T21:15:44.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
520	2024-12-06T21:15:44.426Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
521	2024-12-06T21:15:44.426Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.77249ms iteration = 6 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
522	2024-12-06T21:15:44.442Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 36.173613ms iteration = 6 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
523	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 3 reason = Timeout
524	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 2 reason = Timeout
525	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
526	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 3 reason = Timeout
527	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
528	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 3 reason = Timeout
529	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 2 reason = Timeout
530	2024-12-06T21:15:54.406Z	WARN	nexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0
531	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
532	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
533	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 19.85\u{b5}s iteration = 3 status = {"enabled":false,"target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0"}
534	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 3 reason = Timeout
535	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 3 reason = Timeout
536	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 3 reason = Timeout
537	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 2 reason = Timeout
538	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
539	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 3 reason = Timeout
540	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
541	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
542	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
543	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 7 reason = Timeout
544	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
545	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 7 reason = Timeout
546	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 3 reason = Timeout
547	2024-12-06T21:15:54.406Z	WARN	nexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:69 reason = no config
548	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
549	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 10.89\u{b5}s iteration = 3 status = {"error":"no config"}
550	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 3 reason = Timeout
551	2024-12-06T21:15:54.406Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
552	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 3 reason = Timeout
553	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
554	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
555	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 3 reason = Timeout
556	2024-12-06T21:15:54.406Z	WARN	nexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:69 reason = no config
557	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 7.38\u{b5}s iteration = 3 status = {"error":"no config"}
558	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = external_endpoints iteration = 2 reason = Timeout
559	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 3 reason = Timeout
560	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
561	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
562	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 2 reason = Timeout
563	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 2 reason = Timeout
564	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
565	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 3 reason = Timeout
566	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
567	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 2 reason = Timeout
568	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
569	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
570	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 2 reason = Timeout
571	2024-12-06T21:15:54.406Z	INFO	nexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-06 21:05:54.406338291 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
572	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
573	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 3 reason = Timeout
574	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
575	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 2 reason = Timeout
576	2024-12-06T21:15:54.406Z	INFO	nexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
577	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 4 reason = Timeout
578	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
579	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
580	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 3 reason = Timeout
581	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
582	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 3 reason = Timeout
583	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
584	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 2 reason = Timeout
585	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 3 reason = Timeout
586	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
587	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
588	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
589	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
590	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
591	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
592	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
593	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
594	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
595	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
596	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
597	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
598	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
599	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
600	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
601	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
602	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
603	2024-12-06T21:15:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
604	2024-12-06T21:15:54.406Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("709da000-e0da-4b29-8160-3112fd080927.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a03854a6-7a78-402b-93b4-65c6d36bf236.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a03854a6-7a78-402b-93b4-65c6d36bf236.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 869, tv_nsec: 905590261 } })
605	2024-12-06T21:15:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
606	2024-12-06T21:15:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
607	2024-12-06T21:15:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
608	2024-12-06T21:15:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
609	2024-12-06T21:15:54.407Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4aff1ae3-2d71-40bf-bc46-79a7276da3c2.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4dccdb26-9cf9-40aa-9229-5d35905cf0e1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("79d475dc-44ec-44f4-9e51-ea8a753c39f3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("79d475dc-44ec-44f4-9e51-ea8a753c39f3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 869, tv_nsec: 905602692 } })
610	2024-12-06T21:15:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
611	2024-12-06T21:15:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
612	2024-12-06T21:15:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
613	2024-12-06T21:15:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
614	2024-12-06T21:15:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
615	2024-12-06T21:15:54.407Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
616	2024-12-06T21:15:54.407Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.184079ms iteration = 2 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
617	2024-12-06T21:15:54.407Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
618	2024-12-06T21:15:54.407Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.567806ms iteration = 2 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
619	2024-12-06T21:15:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
620	2024-12-06T21:15:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
621	2024-12-06T21:15:54.409Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.690611ms iteration = 3 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
622	2024-12-06T21:15:54.409Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
623	2024-12-06T21:15:54.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
624	2024-12-06T21:15:54.410Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
625	2024-12-06T21:15:54.411Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 869, tv_nsec: 909809961 } })
626	2024-12-06T21:15:54.411Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
627	2024-12-06T21:15:54.411Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
628	2024-12-06T21:15:54.411Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
629	2024-12-06T21:15:54.412Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
630	2024-12-06T21:15:54.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
631	2024-12-06T21:15:54.414Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
632	2024-12-06T21:15:54.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
633	2024-12-06T21:15:54.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
634	2024-12-06T21:15:54.414Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
635	2024-12-06T21:15:54.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
636	2024-12-06T21:15:54.415Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
637	2024-12-06T21:15:54.415Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
638	2024-12-06T21:15:54.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
639	2024-12-06T21:15:54.415Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
640	2024-12-06T21:15:54.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
641	2024-12-06T21:15:54.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
642	2024-12-06T21:15:54.416Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
643	2024-12-06T21:15:54.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
644	2024-12-06T21:15:54.416Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
645	2024-12-06T21:15:54.417Z	DEBG	nexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 11.046481ms iteration = 2 status = {"errors":[],"found_port_ok":[]}
646	2024-12-06T21:15:54.417Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 11.527359ms iteration = 3 status = {"errors":[],"garbage_collect_requested":[]}
647	2024-12-06T21:15:54.418Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
648	2024-12-06T21:15:54.418Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
649	2024-12-06T21:15:54.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
650	2024-12-06T21:15:54.421Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
651	2024-12-06T21:15:54.422Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
652	2024-12-06T21:15:54.422Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
653	2024-12-06T21:15:54.422Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
654	2024-12-06T21:15:54.422Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
655	2024-12-06T21:15:54.423Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
656	2024-12-06T21:15:54.424Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
657	2024-12-06T21:15:54.424Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
658	2024-12-06T21:15:54.424Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
659	2024-12-06T21:15:54.424Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
660	2024-12-06T21:15:54.425Z	INFO	nexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-06 21:05:54.406338291 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
661	2024-12-06T21:15:54.425Z	DEBG	nexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 18.934591ms iteration = 2 status = {"expiration":"2024-12-06T21:05:54.406338291Z","pruned":[]}
662	2024-12-06T21:15:54.426Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
663	2024-12-06T21:15:54.426Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
664	2024-12-06T21:15:54.426Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
665	2024-12-06T21:15:54.427Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
666	2024-12-06T21:15:54.427Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
667	2024-12-06T21:15:54.427Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
668	2024-12-06T21:15:54.429Z	DEBG	nexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
669	2024-12-06T21:15:54.429Z	INFO	nexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
670	2024-12-06T21:15:54.429Z	DEBG	nexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 22.858215ms iteration = 2 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
671	2024-12-06T21:15:54.430Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 869, tv_nsec: 928694570 } })
672	2024-12-06T21:15:54.430Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
673	2024-12-06T21:15:54.431Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
674	2024-12-06T21:15:54.431Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
675	2024-12-06T21:15:54.431Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
676	2024-12-06T21:15:54.433Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 869, tv_nsec: 931911893 } })
677	2024-12-06T21:15:54.433Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
678	2024-12-06T21:15:54.433Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
679	2024-12-06T21:15:54.433Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
680	2024-12-06T21:15:54.433Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
681	2024-12-06T21:15:54.433Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
682	2024-12-06T21:15:54.433Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
683	2024-12-06T21:15:54.435Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
684	2024-12-06T21:15:54.436Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
685	2024-12-06T21:15:54.436Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
686	2024-12-06T21:15:54.437Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
687	2024-12-06T21:15:54.439Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
688	2024-12-06T21:15:54.441Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
689	2024-12-06T21:15:54.441Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
690	2024-12-06T21:15:54.441Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
691	2024-12-06T21:15:54.459Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
692	2024-12-06T21:15:54.460Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
693	2024-12-06T21:15:54.465Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
694	2024-12-06T21:15:54.465Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
695	2024-12-06T21:15:54.466Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "56ae345d-3e07-4c4e-b8f5-10bbeeb423e6", "content-length": "26", "date": "Fri, 06 Dec 2024 21:15:54 GMT"} })
696	2024-12-06T21:15:54.466Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
697	2024-12-06T21:15:54.467Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
698	2024-12-06T21:15:54.469Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 63.030143ms iteration = 3 status = {"errors":[],"records_set_to_done":[]}
699	2024-12-06T21:15:54.469Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
700	2024-12-06T21:15:54.471Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
701	2024-12-06T21:15:54.472Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
702	2024-12-06T21:15:54.472Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
703	2024-12-06T21:15:54.472Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
704	2024-12-06T21:15:54.473Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
705	2024-12-06T21:15:54.473Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
706	2024-12-06T21:15:54.473Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
707	2024-12-06T21:15:54.474Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
708	2024-12-06T21:15:54.474Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
709	2024-12-06T21:15:54.474Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
710	2024-12-06T21:15:54.474Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
711	2024-12-06T21:15:54.474Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 68.678126ms iteration = 7 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
712	2024-12-06T21:15:54.475Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
713	2024-12-06T21:15:54.477Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
714	2024-12-06T21:15:54.482Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
715	2024-12-06T21:15:54.483Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
716	2024-12-06T21:15:54.483Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
717	2024-12-06T21:15:54.484Z	DEBG	nexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
718	2024-12-06T21:15:54.484Z	INFO	nexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
719	2024-12-06T21:15:54.484Z	DEBG	nexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 77.688484ms iteration = 2 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
720	2024-12-06T21:15:54.484Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
721	2024-12-06T21:15:54.484Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 78.285044ms iteration = 3 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
722	2024-12-06T21:15:54.484Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
723	2024-12-06T21:15:54.484Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
724	2024-12-06T21:15:54.484Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
725	2024-12-06T21:15:54.503Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
726	2024-12-06T21:15:54.503Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 97.406728ms iteration = 7 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
727	2024-12-06T21:15:54.508Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
728	2024-12-06T21:15:54.508Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
729	2024-12-06T21:15:54.510Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
730	2024-12-06T21:15:54.510Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
731	2024-12-06T21:15:54.510Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
732	2024-12-06T21:15:54.510Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
733	2024-12-06T21:15:54.510Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
734	2024-12-06T21:15:54.510Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
735	2024-12-06T21:15:54.510Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
736	2024-12-06T21:15:54.510Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
737	2024-12-06T21:15:54.510Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
738	2024-12-06T21:15:54.510Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 104.431813ms iteration = 3 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
739	2024-12-06T21:15:54.512Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
740	2024-12-06T21:15:54.515Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
741	2024-12-06T21:15:54.515Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
742	2024-12-06T21:15:54.515Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
743	2024-12-06T21:15:54.515Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
744	2024-12-06T21:15:54.515Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
745	2024-12-06T21:15:54.516Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
746	2024-12-06T21:15:54.516Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
747	2024-12-06T21:15:54.516Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
748	2024-12-06T21:15:54.516Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
749	2024-12-06T21:15:54.517Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
750	2024-12-06T21:15:54.517Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
751	2024-12-06T21:15:54.518Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
752	2024-12-06T21:15:54.518Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 05c53cbc-f66c-491f-af8d-6f291fe21596 dns_zone_name = oxide.test found_so_far = 0 version = 2
753	2024-12-06T21:15:54.518Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
754	2024-12-06T21:15:54.518Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
755	2024-12-06T21:15:54.518Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
756	2024-12-06T21:15:54.519Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
757	2024-12-06T21:15:54.519Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
758	2024-12-06T21:15:54.521Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
759	2024-12-06T21:15:54.521Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
760	2024-12-06T21:15:54.521Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
761	2024-12-06T21:15:54.521Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
762	2024-12-06T21:15:54.522Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
763	2024-12-06T21:15:54.522Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
764	2024-12-06T21:15:54.522Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
765	2024-12-06T21:15:54.522Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
766	2024-12-06T21:15:54.523Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
767	2024-12-06T21:15:54.523Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 93c3f865-bc00-42df-bef7-20e39342f84f dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
768	2024-12-06T21:15:54.523Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
769	2024-12-06T21:15:54.523Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
770	2024-12-06T21:15:54.524Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 117.904284ms iteration = 3 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
771	2024-12-06T21:15:54.524Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 05c53cbc-f66c-491f-af8d-6f291fe21596 dns_zone_name = oxide.test found_so_far = 1 version = 2
772	2024-12-06T21:15:54.524Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
773	2024-12-06T21:15:54.524Z	INFO	nexus (BackgroundTasks): found latest generation (first find) background_task = dns_config_external dns_group = external file = nexus/src/app/background/tasks/dns_config.rs:87 generation = 2
774	2024-12-06T21:15:54.524Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 118.225069ms iteration = 2 status = {"generation":2}
775	2024-12-06T21:15:54.524Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 4 reason = Dependency
776	2024-12-06T21:15:54.525Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 118.672126ms iteration = 3 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
777	2024-12-06T21:15:54.527Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
778	2024-12-06T21:15:54.531Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
779	2024-12-06T21:15:54.531Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
780	2024-12-06T21:15:54.531Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
781	2024-12-06T21:15:54.534Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 93c3f865-bc00-42df-bef7-20e39342f84f dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
782	2024-12-06T21:15:54.534Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
783	2024-12-06T21:15:54.534Z	INFO	nexus (BackgroundTasks): found latest generation (first find) background_task = dns_config_internal dns_group = internal file = nexus/src/app/background/tasks/dns_config.rs:87 generation = 1
784	2024-12-06T21:15:54.534Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 127.932718ms iteration = 2 status = {"generation":1}
785	2024-12-06T21:15:54.534Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 4 reason = Dependency
786	2024-12-06T21:15:54.547Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 140.887951ms iteration = 3 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
787	2024-12-06T21:15:54.560Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
788	2024-12-06T21:15:54.560Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
789	2024-12-06T21:15:54.577Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 170.948604ms iteration = 4 status = {"physical_disks_added":0}
790	2024-12-06T21:15:54.579Z	WARN	nexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
791	2024-12-06T21:15:54.579Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 173.676939ms iteration = 2 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
792	2024-12-06T21:15:54.599Z	INFO	nexus (BackgroundTasks): found Silo/DNS/TLS config (initial) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-06T21:14:50.566199Z, time_modified: 2024-12-06T21:14:50.566199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, db_silo: Silo { identity: SiloIdentity { id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-06T21:15:06.991365Z, time_modified: 2024-12-06T21:15:06.991365Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, db_silo: Silo { identity: SiloIdentity { id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-06T21:15:06.991365Z, time_modified: 2024-12-06T21:15:06.991365Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:77
793	2024-12-06T21:15:54.599Z	DEBG	nexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 192.881994ms iteration = 2 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"683d02e1-6ec8-48bf-98d0-db6c94785f80","tls_certs":["d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1"]}},"default_endpoint":{"silo_id":"683d02e1-6ec8-48bf-98d0-db6c94785f80","tls_certs":["d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
794	2024-12-06T21:15:54.607Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
795	2024-12-06T21:15:54.608Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
796	2024-12-06T21:15:54.609Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
797	2024-12-06T21:15:54.611Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
798	2024-12-06T21:15:54.611Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
799	2024-12-06T21:15:54.611Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
800	2024-12-06T21:15:54.612Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
801	2024-12-06T21:15:54.615Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
802	2024-12-06T21:15:54.615Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
803	2024-12-06T21:15:54.615Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
804	2024-12-06T21:15:54.615Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
805	2024-12-06T21:15:54.615Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b3e4b32-acd8-4a73-8d6b-bed1bd9a269f", "content-length": "1", "date": "Fri, 06 Dec 2024 21:15:53 GMT"} })
806	2024-12-06T21:15:54.617Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
807	2024-12-06T21:15:54.618Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
808	2024-12-06T21:15:54.620Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 214.481178ms iteration = 3 status = null
809	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = / versions = all
810	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /access versions = all
811	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /assets/{path} versions = all
812	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /device/auth versions = all
813	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /device/confirm versions = all
814	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /device/success versions = all
815	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /device/token versions = all
816	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /device/verify versions = all
817	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /experimental/v1/probes versions = all
818	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /experimental/v1/probes versions = all
819	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /experimental/v1/probes/{probe} versions = all
820	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /experimental/v1/probes/{probe} versions = all
821	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /images versions = all
822	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /login versions = all
823	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /login/{silo_name}/local versions = all
824	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /login/{silo_name}/saml/{provider_name} versions = all
825	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /login/{silo_name}/saml/{provider_name} versions = all
826	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect versions = all
827	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /lookup/{path} versions = all
828	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /projects/{path} versions = all
829	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /projects-new versions = all
830	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /settings/{path} versions = all
831	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /system/{path} versions = all
832	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /utilization versions = all
833	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/certificates versions = all
834	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/certificates versions = all
835	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/certificates/{certificate} versions = all
836	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/certificates/{certificate} versions = all
837	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/disks versions = all
838	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/disks versions = all
839	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/disks/{disk} versions = all
840	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/disks/{disk} versions = all
841	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/disks/{disk}/bulk-write versions = all
842	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/disks/{disk}/bulk-write-start versions = all
843	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/disks/{disk}/bulk-write-stop versions = all
844	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/disks/{disk}/finalize versions = all
845	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/disks/{disk}/metrics/{metric} versions = all
846	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/floating-ips versions = all
847	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/floating-ips versions = all
848	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/floating-ips/{floating_ip} versions = all
849	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/floating-ips/{floating_ip} versions = all
850	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/floating-ips/{floating_ip} versions = all
851	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/floating-ips/{floating_ip}/attach versions = all
852	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/floating-ips/{floating_ip}/detach versions = all
853	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/groups versions = all
854	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/groups/{group_id} versions = all
855	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/images versions = all
856	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/images versions = all
857	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/images/{image} versions = all
858	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/images/{image} versions = all
859	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/images/{image}/demote versions = all
860	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/images/{image}/promote versions = all
861	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances versions = all
862	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/instances versions = all
863	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/instances/{instance} versions = all
864	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances/{instance} versions = all
865	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/instances/{instance} versions = all
866	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances/{instance}/disks versions = all
867	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/instances/{instance}/disks/attach versions = all
868	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/instances/{instance}/disks/detach versions = all
869	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances/{instance}/external-ips versions = all
870	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral versions = all
871	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/instances/{instance}/external-ips/ephemeral versions = all
872	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/instances/{instance}/reboot versions = all
873	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances/{instance}/serial-console versions = all
874	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances/{instance}/serial-console/stream versions = all
875	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances/{instance}/ssh-public-keys versions = all
876	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/instances/{instance}/start versions = all
877	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/instances/{instance}/stop versions = all
878	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/internet-gateway-ip-addresses versions = all
879	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/internet-gateway-ip-addresses versions = all
880	2024-12-06T21:15:54.645Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/internet-gateway-ip-addresses/{address} versions = all
881	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/internet-gateway-ip-pools versions = all
882	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/internet-gateway-ip-pools versions = all
883	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/internet-gateway-ip-pools/{pool} versions = all
884	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/internet-gateways versions = all
885	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/internet-gateways versions = all
886	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/internet-gateways/{gateway} versions = all
887	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/internet-gateways/{gateway} versions = all
888	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/ip-pools versions = all
889	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/ip-pools/{pool} versions = all
890	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/login/{silo_name}/local versions = all
891	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/logout versions = all
892	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/me versions = all
893	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/me/groups versions = all
894	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/me/ssh-keys versions = all
895	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/me/ssh-keys versions = all
896	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/me/ssh-keys/{ssh_key} versions = all
897	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/me/ssh-keys/{ssh_key} versions = all
898	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/metrics/{metric_name} versions = all
899	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/network-interfaces versions = all
900	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/network-interfaces versions = all
901	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/network-interfaces/{interface} versions = all
902	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/network-interfaces/{interface} versions = all
903	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/network-interfaces/{interface} versions = all
904	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/ping versions = all
905	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/policy versions = all
906	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/policy versions = all
907	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/projects versions = all
908	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/projects versions = all
909	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/projects/{project} versions = all
910	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/projects/{project} versions = all
911	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/projects/{project} versions = all
912	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/projects/{project}/policy versions = all
913	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/projects/{project}/policy versions = all
914	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/snapshots versions = all
915	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/snapshots versions = all
916	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/snapshots/{snapshot} versions = all
917	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/snapshots/{snapshot} versions = all
918	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/disks versions = all
919	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/disks/{disk_id} versions = all
920	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/racks versions = all
921	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/racks/{rack_id} versions = all
922	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/sleds versions = all
923	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/hardware/sleds versions = all
924	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/sleds/{sled_id} versions = all
925	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks versions = all
926	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances versions = all
927	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy versions = all
928	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/sleds-uninitialized versions = all
929	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/switch-port versions = all
930	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings versions = all
931	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/hardware/switch-port/{port}/settings versions = all
932	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/switch-port/{port}/status versions = all
933	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/switches versions = all
934	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/switches/{switch_id} versions = all
935	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/identity-providers versions = all
936	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/identity-providers/local/users versions = all
937	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/identity-providers/local/users/{user_id} versions = all
938	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password versions = all
939	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/identity-providers/saml versions = all
940	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/identity-providers/saml/{provider} versions = all
941	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/ip-pools versions = all
942	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/ip-pools versions = all
943	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/ip-pools/{pool} versions = all
944	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/ip-pools/{pool} versions = all
945	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/ip-pools/{pool} versions = all
946	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/ip-pools/{pool}/ranges versions = all
947	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/ip-pools/{pool}/ranges/add versions = all
948	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove versions = all
949	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/ip-pools/{pool}/silos versions = all
950	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/ip-pools/{pool}/silos versions = all
951	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
952	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
953	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/ip-pools/{pool}/utilization versions = all
954	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/ip-pools-service versions = all
955	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/ip-pools-service/ranges versions = all
956	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/ip-pools-service/ranges/add versions = all
957	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/ip-pools-service/ranges/remove versions = all
958	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/metrics/{metric_name} versions = all
959	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/address-lot versions = all
960	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/networking/address-lot versions = all
961	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/networking/address-lot/{address_lot} versions = all
962	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks versions = all
963	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/allow-list versions = all
964	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/networking/allow-list versions = all
965	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/networking/bfd-disable versions = all
966	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/networking/bfd-enable versions = all
967	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bfd-status versions = all
968	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/networking/bgp versions = all
969	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bgp versions = all
970	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/networking/bgp versions = all
971	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bgp-announce-set versions = all
972	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/networking/bgp-announce-set versions = all
973	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/networking/bgp-announce-set/{announce_set} versions = all
974	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bgp-announce-set/{announce_set}/announcement versions = all
975	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bgp-exported versions = all
976	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bgp-message-history versions = all
977	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bgp-routes-ipv4 versions = all
978	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bgp-status versions = all
979	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/loopback-address versions = all
980	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/networking/loopback-address versions = all
981	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask} versions = all
982	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/networking/switch-port-settings versions = all
983	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/switch-port-settings versions = all
984	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/networking/switch-port-settings versions = all
985	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/switch-port-settings/{port} versions = all
986	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/policy versions = all
987	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/policy versions = all
988	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/roles versions = all
989	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/roles/{role_name} versions = all
990	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/silo-quotas versions = all
991	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/silos versions = all
992	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/silos versions = all
993	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/silos/{silo} versions = all
994	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/silos/{silo} versions = all
995	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/silos/{silo}/ip-pools versions = all
996	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/silos/{silo}/policy versions = all
997	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/silos/{silo}/policy versions = all
998	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/silos/{silo}/quotas versions = all
999	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/silos/{silo}/quotas versions = all
1000	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/timeseries/query versions = all
1001	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/timeseries/schemas versions = all
1002	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/update/repository versions = all
1003	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/update/repository/{system_version} versions = all
1004	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/users versions = all
1005	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/users/{user_id} versions = all
1006	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/users-builtin versions = all
1007	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/users-builtin/{user} versions = all
1008	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/utilization/silos versions = all
1009	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/utilization/silos/{silo} versions = all
1010	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/timeseries/query versions = all
1011	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/users versions = all
1012	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/utilization versions = all
1013	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-firewall-rules versions = all
1014	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/vpc-firewall-rules versions = all
1015	2024-12-06T21:15:54.646Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-router-routes versions = all
1016	2024-12-06T21:15:54.647Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/vpc-router-routes versions = all
1017	2024-12-06T21:15:54.647Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/vpc-router-routes/{route} versions = all
1018	2024-12-06T21:15:54.647Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-router-routes/{route} versions = all
1019	2024-12-06T21:15:54.647Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/vpc-router-routes/{route} versions = all
1020	2024-12-06T21:15:54.647Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-routers versions = all
1021	2024-12-06T21:15:54.647Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/vpc-routers versions = all
1022	2024-12-06T21:15:54.647Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/vpc-routers/{router} versions = all
1023	2024-12-06T21:15:54.647Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-routers/{router} versions = all
1024	2024-12-06T21:15:54.647Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/vpc-routers/{router} versions = all
1025	2024-12-06T21:15:54.647Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-subnets versions = all
1026	2024-12-06T21:15:54.647Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/vpc-subnets versions = all
1027	2024-12-06T21:15:54.647Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/vpc-subnets/{subnet} versions = all
1028	2024-12-06T21:15:54.647Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-subnets/{subnet} versions = all
1029	2024-12-06T21:15:54.647Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/vpc-subnets/{subnet} versions = all
1030	2024-12-06T21:15:54.647Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces versions = all
1031	2024-12-06T21:15:54.647Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpcs versions = all
1032	2024-12-06T21:15:54.647Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/vpcs versions = all
1033	2024-12-06T21:15:54.647Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/vpcs/{vpc} versions = all
1034	2024-12-06T21:15:54.647Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpcs/{vpc} versions = all
1035	2024-12-06T21:15:54.647Z	DEBG	nexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/vpcs/{vpc} versions = all
1036	2024-12-06T21:15:54.647Z	INFO	nexus (dropshot_external): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = 172.30.2.7:443
1037	2024-12-06T21:15:54.647Z	DEBG	nexus (dropshot_external): successfully registered DTrace USDT probes local_addr = 172.30.2.7:443
1038	2024-12-06T21:15:54.658Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = / versions = all
1039	2024-12-06T21:15:54.658Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /access versions = all
1040	2024-12-06T21:15:54.658Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /assets/{path} versions = all
1041	2024-12-06T21:15:54.658Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /device/auth versions = all
1042	2024-12-06T21:15:54.658Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /device/confirm versions = all
1043	2024-12-06T21:15:54.658Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /device/success versions = all
1044	2024-12-06T21:15:54.658Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /device/token versions = all
1045	2024-12-06T21:15:54.658Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /device/verify versions = all
1046	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /experimental/v1/probes versions = all
1047	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /experimental/v1/probes versions = all
1048	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /experimental/v1/probes/{probe} versions = all
1049	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /experimental/v1/probes/{probe} versions = all
1050	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /images versions = all
1051	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /login versions = all
1052	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /login/{silo_name}/local versions = all
1053	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /login/{silo_name}/saml/{provider_name} versions = all
1054	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /login/{silo_name}/saml/{provider_name} versions = all
1055	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect versions = all
1056	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /lookup/{path} versions = all
1057	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /projects/{path} versions = all
1058	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /projects-new versions = all
1059	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /settings/{path} versions = all
1060	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /system/{path} versions = all
1061	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /utilization versions = all
1062	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/certificates versions = all
1063	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/certificates versions = all
1064	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/certificates/{certificate} versions = all
1065	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/certificates/{certificate} versions = all
1066	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/disks versions = all
1067	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/disks versions = all
1068	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/disks/{disk} versions = all
1069	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/disks/{disk} versions = all
1070	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/disks/{disk}/bulk-write versions = all
1071	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/disks/{disk}/bulk-write-start versions = all
1072	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/disks/{disk}/bulk-write-stop versions = all
1073	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/disks/{disk}/finalize versions = all
1074	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/disks/{disk}/metrics/{metric} versions = all
1075	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/floating-ips versions = all
1076	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/floating-ips versions = all
1077	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/floating-ips/{floating_ip} versions = all
1078	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/floating-ips/{floating_ip} versions = all
1079	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/floating-ips/{floating_ip} versions = all
1080	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/floating-ips/{floating_ip}/attach versions = all
1081	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/floating-ips/{floating_ip}/detach versions = all
1082	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/groups versions = all
1083	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/groups/{group_id} versions = all
1084	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/images versions = all
1085	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/images versions = all
1086	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/images/{image} versions = all
1087	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/images/{image} versions = all
1088	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/images/{image}/demote versions = all
1089	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/images/{image}/promote versions = all
1090	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances versions = all
1091	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/instances versions = all
1092	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/instances/{instance} versions = all
1093	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances/{instance} versions = all
1094	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/instances/{instance} versions = all
1095	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances/{instance}/disks versions = all
1096	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/instances/{instance}/disks/attach versions = all
1097	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/instances/{instance}/disks/detach versions = all
1098	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances/{instance}/external-ips versions = all
1099	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral versions = all
1100	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/instances/{instance}/external-ips/ephemeral versions = all
1101	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/instances/{instance}/reboot versions = all
1102	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances/{instance}/serial-console versions = all
1103	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances/{instance}/serial-console/stream versions = all
1104	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances/{instance}/ssh-public-keys versions = all
1105	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/instances/{instance}/start versions = all
1106	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/instances/{instance}/stop versions = all
1107	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/internet-gateway-ip-addresses versions = all
1108	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/internet-gateway-ip-addresses versions = all
1109	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/internet-gateway-ip-addresses/{address} versions = all
1110	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/internet-gateway-ip-pools versions = all
1111	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/internet-gateway-ip-pools versions = all
1112	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/internet-gateway-ip-pools/{pool} versions = all
1113	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/internet-gateways versions = all
1114	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/internet-gateways versions = all
1115	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/internet-gateways/{gateway} versions = all
1116	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/internet-gateways/{gateway} versions = all
1117	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/ip-pools versions = all
1118	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/ip-pools/{pool} versions = all
1119	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/login/{silo_name}/local versions = all
1120	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/logout versions = all
1121	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/me versions = all
1122	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/me/groups versions = all
1123	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/me/ssh-keys versions = all
1124	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/me/ssh-keys versions = all
1125	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/me/ssh-keys/{ssh_key} versions = all
1126	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/me/ssh-keys/{ssh_key} versions = all
1127	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/metrics/{metric_name} versions = all
1128	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/network-interfaces versions = all
1129	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/network-interfaces versions = all
1130	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/network-interfaces/{interface} versions = all
1131	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/network-interfaces/{interface} versions = all
1132	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/network-interfaces/{interface} versions = all
1133	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/ping versions = all
1134	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/policy versions = all
1135	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/policy versions = all
1136	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/projects versions = all
1137	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/projects versions = all
1138	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/projects/{project} versions = all
1139	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/projects/{project} versions = all
1140	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/projects/{project} versions = all
1141	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/projects/{project}/policy versions = all
1142	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/projects/{project}/policy versions = all
1143	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/snapshots versions = all
1144	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/snapshots versions = all
1145	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/snapshots/{snapshot} versions = all
1146	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/snapshots/{snapshot} versions = all
1147	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/disks versions = all
1148	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/disks/{disk_id} versions = all
1149	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/racks versions = all
1150	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/racks/{rack_id} versions = all
1151	2024-12-06T21:15:54.659Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/sleds versions = all
1152	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/hardware/sleds versions = all
1153	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/sleds/{sled_id} versions = all
1154	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks versions = all
1155	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances versions = all
1156	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy versions = all
1157	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/sleds-uninitialized versions = all
1158	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/switch-port versions = all
1159	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings versions = all
1160	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/hardware/switch-port/{port}/settings versions = all
1161	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/switch-port/{port}/status versions = all
1162	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/switches versions = all
1163	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/switches/{switch_id} versions = all
1164	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/identity-providers versions = all
1165	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/identity-providers/local/users versions = all
1166	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/identity-providers/local/users/{user_id} versions = all
1167	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password versions = all
1168	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/identity-providers/saml versions = all
1169	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/identity-providers/saml/{provider} versions = all
1170	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/ip-pools versions = all
1171	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/ip-pools versions = all
1172	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/ip-pools/{pool} versions = all
1173	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/ip-pools/{pool} versions = all
1174	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/ip-pools/{pool} versions = all
1175	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/ip-pools/{pool}/ranges versions = all
1176	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/ip-pools/{pool}/ranges/add versions = all
1177	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove versions = all
1178	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/ip-pools/{pool}/silos versions = all
1179	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/ip-pools/{pool}/silos versions = all
1180	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
1181	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
1182	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/ip-pools/{pool}/utilization versions = all
1183	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/ip-pools-service versions = all
1184	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/ip-pools-service/ranges versions = all
1185	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/ip-pools-service/ranges/add versions = all
1186	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/ip-pools-service/ranges/remove versions = all
1187	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/metrics/{metric_name} versions = all
1188	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/address-lot versions = all
1189	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/networking/address-lot versions = all
1190	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/networking/address-lot/{address_lot} versions = all
1191	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks versions = all
1192	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/allow-list versions = all
1193	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/networking/allow-list versions = all
1194	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/networking/bfd-disable versions = all
1195	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/networking/bfd-enable versions = all
1196	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bfd-status versions = all
1197	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/networking/bgp versions = all
1198	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bgp versions = all
1199	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/networking/bgp versions = all
1200	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bgp-announce-set versions = all
1201	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/networking/bgp-announce-set versions = all
1202	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/networking/bgp-announce-set/{announce_set} versions = all
1203	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bgp-announce-set/{announce_set}/announcement versions = all
1204	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bgp-exported versions = all
1205	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bgp-message-history versions = all
1206	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bgp-routes-ipv4 versions = all
1207	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bgp-status versions = all
1208	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/loopback-address versions = all
1209	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/networking/loopback-address versions = all
1210	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask} versions = all
1211	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/networking/switch-port-settings versions = all
1212	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/switch-port-settings versions = all
1213	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/networking/switch-port-settings versions = all
1214	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/switch-port-settings/{port} versions = all
1215	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/policy versions = all
1216	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/policy versions = all
1217	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/roles versions = all
1218	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/roles/{role_name} versions = all
1219	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/silo-quotas versions = all
1220	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/silos versions = all
1221	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/silos versions = all
1222	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/silos/{silo} versions = all
1223	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/silos/{silo} versions = all
1224	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/silos/{silo}/ip-pools versions = all
1225	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/silos/{silo}/policy versions = all
1226	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/silos/{silo}/policy versions = all
1227	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/silos/{silo}/quotas versions = all
1228	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/silos/{silo}/quotas versions = all
1229	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/timeseries/query versions = all
1230	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/timeseries/schemas versions = all
1231	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/update/repository versions = all
1232	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/update/repository/{system_version} versions = all
1233	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/users versions = all
1234	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/users/{user_id} versions = all
1235	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/users-builtin versions = all
1236	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/users-builtin/{user} versions = all
1237	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/utilization/silos versions = all
1238	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/utilization/silos/{silo} versions = all
1239	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/timeseries/query versions = all
1240	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/users versions = all
1241	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/utilization versions = all
1242	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-firewall-rules versions = all
1243	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/vpc-firewall-rules versions = all
1244	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-router-routes versions = all
1245	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/vpc-router-routes versions = all
1246	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/vpc-router-routes/{route} versions = all
1247	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-router-routes/{route} versions = all
1248	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/vpc-router-routes/{route} versions = all
1249	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-routers versions = all
1250	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/vpc-routers versions = all
1251	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/vpc-routers/{router} versions = all
1252	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-routers/{router} versions = all
1253	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/vpc-routers/{router} versions = all
1254	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-subnets versions = all
1255	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/vpc-subnets versions = all
1256	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/vpc-subnets/{subnet} versions = all
1257	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-subnets/{subnet} versions = all
1258	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/vpc-subnets/{subnet} versions = all
1259	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces versions = all
1260	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpcs versions = all
1261	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/vpcs versions = all
1262	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/vpcs/{vpc} versions = all
1263	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpcs/{vpc} versions = all
1264	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/vpcs/{vpc} versions = all
1265	2024-12-06T21:15:54.660Z	INFO	nexus (dropshot_external_techport): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = [fd00:1122:3344:101::c]:12228
1266	2024-12-06T21:15:54.660Z	DEBG	nexus (dropshot_external_techport): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::c]:12228
1267	2024-12-06T21:15:54.665Z	DEBG	nexus (nexus-producer-server): registered DTrace probes
1268	2024-12-06T21:15:54.668Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1269	2024-12-06T21:15:54.668Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1270	2024-12-06T21:15:54.671Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1271	2024-12-06T21:15:54.671Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1272	2024-12-06T21:15:54.673Z	DEBG	nexus (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::c]:52742 method = GET path = /{producer_id} versions = all
1273	2024-12-06T21:15:54.673Z	INFO	nexus (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = [fd00:1122:3344:101::c]:52742
1274	2024-12-06T21:15:54.673Z	DEBG	nexus (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::c]:52742
1275	2024-12-06T21:15:54.673Z	DEBG	nexus (nexus-producer-server): Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::c]:52742
1276	2024-12-06T21:15:54.673Z	DEBG	nexus (nexus-producer-server): Nexus IP provided explicitly, will use it addr = [fd00:1122:3344:101::c]:12221
1277	2024-12-06T21:15:54.673Z	DEBG	nexus (nexus-producer-server): starting producer registration task
1278	2024-12-06T21:15:54.674Z	INFO	nexus (nexus-producer-server): starting oximeter metric producer server address = [fd00:1122:3344:101::c]:52742 file = oximeter/producer/src/lib.rs:279 interval = 10s producer_id = e8725c8c-13ab-4916-bf16-bb7f832014d5
1279	2024-12-06T21:15:54.674Z	DEBG	nexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
1280	2024-12-06T21:15:54.674Z	DEBG	nexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
1281	2024-12-06T21:15:54.680Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 zone_address = fd00:1122:3344:101::2
1282	2024-12-06T21:15:54.680Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1283	2024-12-06T21:15:54.681Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5b7fb710-4342-4d0b-8523-4636366cc8c5", "content-length": "26", "date": "Fri, 06 Dec 2024 21:15:54 GMT"} })
1284	2024-12-06T21:15:54.681Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1285	2024-12-06T21:15:54.682Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
1286	2024-12-06T21:15:54.704Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
1287	2024-12-06T21:15:54.704Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1288	2024-12-06T21:15:54.704Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "54403737-4119-4418-bc53-8eacefa48df5", "content-length": "26", "date": "Fri, 06 Dec 2024 21:15:54 GMT"} })
1289	2024-12-06T21:15:54.704Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1290	2024-12-06T21:15:54.706Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
1291	2024-12-06T21:15:54.728Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
1292	2024-12-06T21:15:54.741Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
1293	2024-12-06T21:15:54.741Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
1294	2024-12-06T21:15:54.742Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "83d3b40d-4ebe-40bb-870f-83a2fed57d37", "content-length": "372", "date": "Fri, 06 Dec 2024 21:15:54 GMT"} })
1295	2024-12-06T21:15:54.742Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1296	2024-12-06T21:15:54.742Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1297	2024-12-06T21:15:54.747Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
1298	2024-12-06T21:15:54.840Z	DEBG	nexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
1299	2024-12-06T21:15:54.841Z	INFO	nexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:49239
1300	2024-12-06T21:15:54.855Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
1301	2024-12-06T21:15:54.855Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a7471d6-b0ef-497a-8688-5eead7b5d645", "content-length": "691", "date": "Fri, 06 Dec 2024 21:15:54 GMT"} })
1302	2024-12-06T21:15:54.855Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
1303	2024-12-06T21:15:54.855Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
1304	2024-12-06T21:15:54.855Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
1305	2024-12-06T21:15:54.857Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:49239 req_id = f53bda04-52a0-4118-ae8b-d17a9132f58f roles = RoleSet { roles: {} } uri = /metrics/producers
1306	2024-12-06T21:15:54.857Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 451.368623ms iteration = 3 status = {}
1307	2024-12-06T21:15:54.857Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1308	2024-12-06T21:15:54.857Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:49239 req_id = f53bda04-52a0-4118-ae8b-d17a9132f58f resource = Database result = Ok(()) uri = /metrics/producers
1309	2024-12-06T21:15:54.861Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cbc5c1c6-5668-455a-85ef-94f4aebe0f80, lookup_type: ById(cbc5c1c6-5668-455a-85ef-94f4aebe0f80) } result = Ok(())
1310	2024-12-06T21:15:54.861Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1311	2024-12-06T21:15:54.861Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1312	2024-12-06T21:15:54.864Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 457.931341ms iteration = 3 status = 0
1313	2024-12-06T21:15:54.875Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
1314	2024-12-06T21:15:54.880Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 474.443571ms iteration = 3 status = {}
1315	2024-12-06T21:15:54.883Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
1316	2024-12-06T21:15:54.883Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
1317	2024-12-06T21:15:54.883Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "c191a7d7-6467-4614-b27d-db1957343818", "date": "Fri, 06 Dec 2024 21:15:54 GMT"} })
1318	2024-12-06T21:15:54.883Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1319	2024-12-06T21:15:54.884Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1320	2024-12-06T21:15:54.887Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1321	2024-12-06T21:15:54.887Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1322	2024-12-06T21:15:54.890Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1323	2024-12-06T21:15:54.890Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1324	2024-12-06T21:15:54.894Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
1325	2024-12-06T21:15:54.894Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
1326	2024-12-06T21:15:54.894Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "4275d849-fc7e-4db8-a3fe-79f3f5769c47", "date": "Fri, 06 Dec 2024 21:15:54 GMT"} })
1327	2024-12-06T21:15:54.895Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 489.716533ms iteration = 3 status = {}
1328	2024-12-06T21:15:54.906Z	INFO	nexus: assigned collector to new producer collector_id = 9d509eff-9671-46fb-8274-6ec9019a9d9e file = nexus/src/app/oximeter.rs:76 producer_id = e8725c8c-13ab-4916-bf16-bb7f832014d5
1329	2024-12-06T21:15:54.906Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 65144 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:49239 req_id = f53bda04-52a0-4118-ae8b-d17a9132f58f response_code = 201 uri = /metrics/producers
1330	2024-12-06T21:15:54.906Z	DEBG	nexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "f53bda04-52a0-4118-ae8b-d17a9132f58f", "content-length": "41", "date": "Fri, 06 Dec 2024 21:15:54 GMT"} })
1331	2024-12-06T21:15:54.907Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
1332	2024-12-06T21:15:54.908Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "7067f2e3-e55d-4c17-8bb7-b5e9243443d6", "date": "Fri, 06 Dec 2024 21:15:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
1333	2024-12-06T21:15:54.908Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
1334	2024-12-06T21:15:54.909Z	DEBG	nexus (producer-registration-task): registered with nexus successfully
1335	2024-12-06T21:15:54.909Z	DEBG	nexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
1336	2024-12-06T21:15:54.909Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "7bc74293-349c-484d-a7d3-512051f95e44", "date": "Fri, 06 Dec 2024 21:15:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
1337	2024-12-06T21:15:54.909Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
1338	2024-12-06T21:15:54.911Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
1339	2024-12-06T21:15:54.911Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 386.686153ms iteration = 4 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
1340	2024-12-06T21:15:54.954Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
1341	2024-12-06T21:15:54.955Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "9276ac7c-ab38-43fd-b597-073ee86c4fc7", "date": "Fri, 06 Dec 2024 21:15:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
1342	2024-12-06T21:15:54.955Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
1343	2024-12-06T21:15:54.957Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "7f38a411-da68-4552-8b3e-31784e5fd656", "date": "Fri, 06 Dec 2024 21:15:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
1344	2024-12-06T21:15:54.957Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
1345	2024-12-06T21:15:54.958Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "c9fc22f3-fb58-44ea-8a6c-d68bc79de7ea", "date": "Fri, 06 Dec 2024 21:15:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
1346	2024-12-06T21:15:54.958Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
1347	2024-12-06T21:15:54.960Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
1348	2024-12-06T21:15:54.960Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 426.436425ms iteration = 4 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
1349	2024-12-06T21:15:54.982Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1350	2024-12-06T21:15:54.982Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1351	2024-12-06T21:15:54.984Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 3d56b7bf-53ed-4147-b45b-9cef154d7338, rack_id: 2d75e4a3-6363-4fd1-935e-989d31fd7d71, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(02859c9b-92d7-4b19-b425-8f266763be54) } rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
1352	2024-12-06T21:15:54.984Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1353	2024-12-06T21:15:54.984Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1354	2024-12-06T21:15:54.995Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
1355	2024-12-06T21:15:54.995Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "011d2c7d-fb7c-408e-9c42-a48e7a628b73", "content-length": "134", "date": "Fri, 06 Dec 2024 21:15:54 GMT"} })
1356	2024-12-06T21:15:54.995Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
1357	2024-12-06T21:15:54.995Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 02859c9b-92d7-4b19-b425-8f266763be54
1358	2024-12-06T21:15:54.996Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
1359	2024-12-06T21:15:54.996Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
1360	2024-12-06T21:15:54.996Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
1361	2024-12-06T21:15:54.996Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
1362	2024-12-06T21:15:54.996Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
1363	2024-12-06T21:15:54.996Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
1364	2024-12-06T21:15:54.996Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
1365	2024-12-06T21:15:54.996Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
1366	2024-12-06T21:15:54.997Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
1367	2024-12-06T21:15:54.997Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1368	2024-12-06T21:15:54.997Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1369	2024-12-06T21:15:55.001Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
1370	2024-12-06T21:15:55.002Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "bf0365d6-712a-4093-984e-5c3cc5b94fa4", "content-length": "2", "date": "Fri, 06 Dec 2024 21:15:54 GMT"} })
1371	2024-12-06T21:15:55.002Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
1372	2024-12-06T21:15:55.003Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "c83c2198-4694-408a-a2a1-873ef1906fce", "content-length": "52", "date": "Fri, 06 Dec 2024 21:15:54 GMT"} })
1373	2024-12-06T21:15:55.003Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
1374	2024-12-06T21:15:55.003Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
1375	2024-12-06T21:15:55.104Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "b01871fd-12a9-4dcf-ace7-b4b521dc8bf7", "date": "Fri, 06 Dec 2024 21:15:55 GMT"} })
1376	2024-12-06T21:15:55.104Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch = Switch0
1377	2024-12-06T21:15:55.104Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
1378	2024-12-06T21:15:55.105Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "35e9cc5d-2ab0-4a62-a2c4-881f26957976", "content-length": "682", "date": "Fri, 06 Dec 2024 21:15:55 GMT"} })
1379	2024-12-06T21:15:55.105Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1380	2024-12-06T21:15:55.105Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1381	2024-12-06T21:15:55.117Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1382	2024-12-06T21:15:55.117Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1383	2024-12-06T21:15:55.120Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1384	2024-12-06T21:15:55.121Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1385	2024-12-06T21:15:55.122Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1386	2024-12-06T21:15:55.123Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1387	2024-12-06T21:15:55.129Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 723.492477ms iteration = 3 status = {}
1388	2024-12-06T21:16:04.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 8 reason = Timeout
1389	2024-12-06T21:16:04.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 8 reason = Timeout
1390	2024-12-06T21:16:04.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1391	2024-12-06T21:16:04.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1392	2024-12-06T21:16:04.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1393	2024-12-06T21:16:04.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1394	2024-12-06T21:16:04.408Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1395	2024-12-06T21:16:04.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1396	2024-12-06T21:16:04.410Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1397	2024-12-06T21:16:04.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1398	2024-12-06T21:16:04.413Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1399	2024-12-06T21:16:04.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1400	2024-12-06T21:16:04.415Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1401	2024-12-06T21:16:04.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
1402	2024-12-06T21:16:04.417Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1403	2024-12-06T21:16:04.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1404	2024-12-06T21:16:04.435Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1405	2024-12-06T21:16:04.435Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1406	2024-12-06T21:16:04.437Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 30.880617ms iteration = 8 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
1407	2024-12-06T21:16:04.437Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
1408	2024-12-06T21:16:04.437Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 31.270373ms iteration = 8 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
1409	2024-12-06T21:16:06.677Z	INFO	nexus (dropshot): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:52742 remote_addr = [fd00:1122:3344:101::d]:63318
1410	2024-12-06T21:16:06.687Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 9737 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = a5b0ee3e-a81a-436c-af09-8a1a1220b7f3 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
1411	2024-12-06T21:16:14.405Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 9 reason = Timeout
1412	2024-12-06T21:16:14.405Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 9 reason = Timeout
1413	2024-12-06T21:16:14.405Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1414	2024-12-06T21:16:14.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1415	2024-12-06T21:16:14.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1416	2024-12-06T21:16:14.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1417	2024-12-06T21:16:14.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1418	2024-12-06T21:16:14.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1419	2024-12-06T21:16:14.410Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1420	2024-12-06T21:16:14.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1421	2024-12-06T21:16:14.413Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1422	2024-12-06T21:16:14.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1423	2024-12-06T21:16:14.415Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1424	2024-12-06T21:16:14.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
1425	2024-12-06T21:16:14.417Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1426	2024-12-06T21:16:14.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1427	2024-12-06T21:16:14.430Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
1428	2024-12-06T21:16:14.430Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.176616ms iteration = 9 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
1429	2024-12-06T21:16:14.436Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1430	2024-12-06T21:16:14.436Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1431	2024-12-06T21:16:14.438Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.91264ms iteration = 9 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
1432	2024-12-06T21:16:16.540Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 282 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 183b4e8c-1178-467f-a1e5-508fe154c7d5 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
1433	2024-12-06T21:16:21.534Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = 8f0680b8-dfb7-4a43-9786-9c08eb2ec525 roles = RoleSet { roles: {} } uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
1434	2024-12-06T21:16:21.535Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = 8f0680b8-dfb7-4a43-9786-9c08eb2ec525 resource = Database result = Ok(()) uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
1435	2024-12-06T21:16:21.537Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3034 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = 8f0680b8-dfb7-4a43-9786-9c08eb2ec525 response_code = 200 uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
1436	2024-12-06T21:16:21.537Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = c07a214c-e1b4-4a63-8672-3c4dd6b271df roles = RoleSet { roles: {} } uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
1437	2024-12-06T21:16:21.538Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = c07a214c-e1b4-4a63-8672-3c4dd6b271df resource = Database result = Ok(()) uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
1438	2024-12-06T21:16:21.540Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2142 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = c07a214c-e1b4-4a63-8672-3c4dd6b271df response_code = 200 uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
1439	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 4 reason = Timeout
1440	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
1441	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 4 reason = Timeout
1442	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 4 reason = Timeout
1443	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
1444	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
1445	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 4 reason = Timeout
1446	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
1447	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 4 reason = Timeout
1448	2024-12-06T21:16:24.406Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
1449	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1450	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 5 reason = Timeout
1451	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 4 reason = Timeout
1452	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1453	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1454	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 4 reason = Timeout
1455	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
1456	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 4 reason = Timeout
1457	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
1458	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 4 reason = Timeout
1459	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1460	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 4 reason = Timeout
1461	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1462	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
1463	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 4 reason = Timeout
1464	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
1465	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 4 reason = Timeout
1466	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
1467	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 10 reason = Timeout
1468	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1469	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 4 reason = Timeout
1470	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
1471	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 4 reason = Timeout
1472	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 10 reason = Timeout
1473	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1474	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1475	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1476	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
1477	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
1478	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1479	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
1480	2024-12-06T21:16:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
1481	2024-12-06T21:16:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
1482	2024-12-06T21:16:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
1483	2024-12-06T21:16:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1484	2024-12-06T21:16:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1485	2024-12-06T21:16:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1486	2024-12-06T21:16:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1487	2024-12-06T21:16:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1488	2024-12-06T21:16:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1489	2024-12-06T21:16:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
1490	2024-12-06T21:16:24.409Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1491	2024-12-06T21:16:24.410Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 899, tv_nsec: 909757402 } })
1492	2024-12-06T21:16:24.411Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
1493	2024-12-06T21:16:24.411Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
1494	2024-12-06T21:16:24.411Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1495	2024-12-06T21:16:24.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1496	2024-12-06T21:16:24.412Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 5.416819ms iteration = 4 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
1497	2024-12-06T21:16:24.413Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1498	2024-12-06T21:16:24.414Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 7.843408ms iteration = 4 status = {"errors":[],"garbage_collect_requested":[]}
1499	2024-12-06T21:16:24.414Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 899, tv_nsec: 913754188 } })
1500	2024-12-06T21:16:24.415Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1501	2024-12-06T21:16:24.415Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
1502	2024-12-06T21:16:24.416Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1503	2024-12-06T21:16:24.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
1504	2024-12-06T21:16:24.418Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
1505	2024-12-06T21:16:24.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
1506	2024-12-06T21:16:24.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
1507	2024-12-06T21:16:24.418Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1508	2024-12-06T21:16:24.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1509	2024-12-06T21:16:24.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1510	2024-12-06T21:16:24.419Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1511	2024-12-06T21:16:24.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1512	2024-12-06T21:16:24.425Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1513	2024-12-06T21:16:24.427Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
1514	2024-12-06T21:16:24.427Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1515	2024-12-06T21:16:24.427Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1516	2024-12-06T21:16:24.428Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1517	2024-12-06T21:16:24.428Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1518	2024-12-06T21:16:24.430Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1519	2024-12-06T21:16:24.430Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1520	2024-12-06T21:16:24.447Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
1521	2024-12-06T21:16:24.447Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1522	2024-12-06T21:16:24.454Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
1523	2024-12-06T21:16:24.454Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
1524	2024-12-06T21:16:24.458Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1525	2024-12-06T21:16:24.458Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1526	2024-12-06T21:16:24.458Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1527	2024-12-06T21:16:24.459Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 52.628213ms iteration = 4 status = {"errors":[],"records_set_to_done":[]}
1528	2024-12-06T21:16:24.460Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
1529	2024-12-06T21:16:24.460Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1530	2024-12-06T21:16:24.460Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1531	2024-12-06T21:16:24.462Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
1532	2024-12-06T21:16:24.462Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
1533	2024-12-06T21:16:24.463Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 899, tv_nsec: 961995369 } })
1534	2024-12-06T21:16:24.463Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1535	2024-12-06T21:16:24.463Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
1536	2024-12-06T21:16:24.463Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1537	2024-12-06T21:16:24.465Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1538	2024-12-06T21:16:24.465Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1539	2024-12-06T21:16:24.465Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1540	2024-12-06T21:16:24.466Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1541	2024-12-06T21:16:24.468Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
1542	2024-12-06T21:16:24.468Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1543	2024-12-06T21:16:24.468Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1544	2024-12-06T21:16:24.469Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cbc5c1c6-5668-455a-85ef-94f4aebe0f80, lookup_type: ById(cbc5c1c6-5668-455a-85ef-94f4aebe0f80) } result = Ok(())
1545	2024-12-06T21:16:24.469Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1546	2024-12-06T21:16:24.469Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1547	2024-12-06T21:16:24.469Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1548	2024-12-06T21:16:24.470Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1549	2024-12-06T21:16:24.471Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1550	2024-12-06T21:16:24.472Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 65.621126ms iteration = 4 status = 0
1551	2024-12-06T21:16:24.472Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1552	2024-12-06T21:16:24.473Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
1553	2024-12-06T21:16:24.473Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1554	2024-12-06T21:16:24.473Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1555	2024-12-06T21:16:24.474Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
1556	2024-12-06T21:16:24.474Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1557	2024-12-06T21:16:24.474Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1558	2024-12-06T21:16:24.477Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
1559	2024-12-06T21:16:24.477Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 70.956394ms iteration = 4 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
1560	2024-12-06T21:16:24.477Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
1561	2024-12-06T21:16:24.477Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
1562	2024-12-06T21:16:24.477Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
1563	2024-12-06T21:16:24.478Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
1564	2024-12-06T21:16:24.480Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
1565	2024-12-06T21:16:24.480Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
1566	2024-12-06T21:16:24.480Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
1567	2024-12-06T21:16:24.480Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
1568	2024-12-06T21:16:24.481Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
1569	2024-12-06T21:16:24.481Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1570	2024-12-06T21:16:24.482Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
1571	2024-12-06T21:16:24.482Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 76.20197ms iteration = 10 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
1572	2024-12-06T21:16:24.490Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 84.211281ms iteration = 4 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
1573	2024-12-06T21:16:24.493Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 87.298271ms iteration = 10 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
1574	2024-12-06T21:16:24.497Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
1575	2024-12-06T21:16:24.497Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1576	2024-12-06T21:16:24.497Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7fea18c4-1f72-4348-8fdf-0fa9d3f539f1", "content-length": "26", "date": "Fri, 06 Dec 2024 21:16:24 GMT"} })
1577	2024-12-06T21:16:24.498Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1578	2024-12-06T21:16:24.499Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
1579	2024-12-06T21:16:24.501Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 95.409675ms iteration = 4 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
1580	2024-12-06T21:16:24.509Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 103.236223ms iteration = 4 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
1581	2024-12-06T21:16:24.511Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
1582	2024-12-06T21:16:24.511Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
1583	2024-12-06T21:16:24.511Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 104.807199ms iteration = 4 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
1584	2024-12-06T21:16:24.521Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
1585	2024-12-06T21:16:24.521Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1586	2024-12-06T21:16:24.521Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5015df99-d8a6-4507-8afd-7995313db35e", "content-length": "26", "date": "Fri, 06 Dec 2024 21:16:24 GMT"} })
1587	2024-12-06T21:16:24.521Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1588	2024-12-06T21:16:24.523Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
1589	2024-12-06T21:16:24.544Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1590	2024-12-06T21:16:24.545Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1591	2024-12-06T21:16:24.573Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 166.732885ms iteration = 5 status = {"physical_disks_added":0}
1592	2024-12-06T21:16:24.606Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 zone_address = fd00:1122:3344:101::2
1593	2024-12-06T21:16:24.606Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1594	2024-12-06T21:16:24.607Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1268fee2-aa7c-41b3-8dad-6d137178571c", "content-length": "26", "date": "Fri, 06 Dec 2024 21:16:24 GMT"} })
1595	2024-12-06T21:16:24.607Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1596	2024-12-06T21:16:24.608Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
1597	2024-12-06T21:16:24.662Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 256.023009ms iteration = 4 status = {}
1598	2024-12-06T21:16:24.724Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
1599	2024-12-06T21:16:24.724Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "ceffba19-ae89-4ade-97c6-0bc109ce78ea", "content-length": "1", "date": "Fri, 06 Dec 2024 21:16:23 GMT"} })
1600	2024-12-06T21:16:24.726Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
1601	2024-12-06T21:16:24.726Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1602	2024-12-06T21:16:24.727Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
1603	2024-12-06T21:16:24.727Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
1604	2024-12-06T21:16:24.727Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "564b0f13-882d-484a-bfc7-37dd542b378d", "content-length": "372", "date": "Fri, 06 Dec 2024 21:16:23 GMT"} })
1605	2024-12-06T21:16:24.727Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1606	2024-12-06T21:16:24.728Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1607	2024-12-06T21:16:24.729Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 323.028658ms iteration = 4 status = null
1608	2024-12-06T21:16:24.785Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1609	2024-12-06T21:16:24.787Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
1610	2024-12-06T21:16:24.787Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
1611	2024-12-06T21:16:24.787Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "73feee5a-39f3-4788-b1d6-c5686b37b899", "date": "Fri, 06 Dec 2024 21:16:23 GMT"} })
1612	2024-12-06T21:16:24.787Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1613	2024-12-06T21:16:24.788Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1614	2024-12-06T21:16:24.788Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1615	2024-12-06T21:16:24.791Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 3d56b7bf-53ed-4147-b45b-9cef154d7338, rack_id: 2d75e4a3-6363-4fd1-935e-989d31fd7d71, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(02859c9b-92d7-4b19-b425-8f266763be54) } rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
1616	2024-12-06T21:16:24.791Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1617	2024-12-06T21:16:24.791Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1618	2024-12-06T21:16:24.791Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1619	2024-12-06T21:16:24.791Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1620	2024-12-06T21:16:24.795Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1621	2024-12-06T21:16:24.795Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1622	2024-12-06T21:16:24.799Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
1623	2024-12-06T21:16:24.799Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
1624	2024-12-06T21:16:24.799Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "92a1c117-0253-4969-b76e-3ee4fdb54d45", "date": "Fri, 06 Dec 2024 21:16:23 GMT"} })
1625	2024-12-06T21:16:24.801Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 394.758914ms iteration = 4 status = {}
1626	2024-12-06T21:16:24.833Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1627	2024-12-06T21:16:24.833Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1628	2024-12-06T21:16:24.835Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1629	2024-12-06T21:16:24.837Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
1630	2024-12-06T21:16:24.837Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1631	2024-12-06T21:16:24.837Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1632	2024-12-06T21:16:24.838Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1633	2024-12-06T21:16:24.840Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
1634	2024-12-06T21:16:24.840Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1635	2024-12-06T21:16:24.840Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1636	2024-12-06T21:16:24.873Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
1637	2024-12-06T21:16:24.873Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "e5a60b50-1c95-4f66-9b92-e9b82b73f00a", "content-length": "691", "date": "Fri, 06 Dec 2024 21:16:23 GMT"} })
1638	2024-12-06T21:16:24.873Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
1639	2024-12-06T21:16:24.873Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
1640	2024-12-06T21:16:24.873Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
1641	2024-12-06T21:16:24.876Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 469.185854ms iteration = 4 status = {}
1642	2024-12-06T21:16:24.909Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
1643	2024-12-06T21:16:24.909Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "62b83691-4839-42eb-8b0b-0346140bca7e", "content-length": "134", "date": "Fri, 06 Dec 2024 21:16:23 GMT"} })
1644	2024-12-06T21:16:24.909Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
1645	2024-12-06T21:16:24.909Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 02859c9b-92d7-4b19-b425-8f266763be54
1646	2024-12-06T21:16:24.910Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
1647	2024-12-06T21:16:24.910Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
1648	2024-12-06T21:16:24.910Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
1649	2024-12-06T21:16:24.910Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
1650	2024-12-06T21:16:24.910Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
1651	2024-12-06T21:16:24.910Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
1652	2024-12-06T21:16:24.910Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
1653	2024-12-06T21:16:24.910Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
1654	2024-12-06T21:16:24.910Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
1655	2024-12-06T21:16:24.910Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1656	2024-12-06T21:16:24.910Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1657	2024-12-06T21:16:24.918Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
1658	2024-12-06T21:16:24.918Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "e2b625a8-6669-44ee-bcb9-1839a790909a", "content-length": "2", "date": "Fri, 06 Dec 2024 21:16:23 GMT"} })
1659	2024-12-06T21:16:24.918Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
1660	2024-12-06T21:16:24.918Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "45014ad1-ce05-49b7-bd02-31a6c12e3e4c", "content-length": "52", "date": "Fri, 06 Dec 2024 21:16:23 GMT"} })
1661	2024-12-06T21:16:24.918Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
1662	2024-12-06T21:16:24.918Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
1663	2024-12-06T21:16:25.102Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "0e97e3c5-5fc1-4058-80b5-4cebe8969fdd", "date": "Fri, 06 Dec 2024 21:16:25 GMT"} })
1664	2024-12-06T21:16:25.102Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch = Switch0
1665	2024-12-06T21:16:25.102Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
1666	2024-12-06T21:16:25.102Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "016b6ca1-5400-40c4-a2f6-b5168e24bac4", "content-length": "682", "date": "Fri, 06 Dec 2024 21:16:25 GMT"} })
1667	2024-12-06T21:16:25.102Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1668	2024-12-06T21:16:25.103Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1669	2024-12-06T21:16:25.107Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1670	2024-12-06T21:16:25.107Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1671	2024-12-06T21:16:25.111Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1672	2024-12-06T21:16:25.112Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1673	2024-12-06T21:16:25.113Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1674	2024-12-06T21:16:25.114Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1675	2024-12-06T21:16:25.128Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 722.211675ms iteration = 4 status = {}
1676	2024-12-06T21:16:26.540Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 321 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = bf97274b-a9ec-4b5e-a09c-01428e415cbb response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
1677	2024-12-06T21:16:34.405Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 11 reason = Timeout
1678	2024-12-06T21:16:34.405Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 11 reason = Timeout
1679	2024-12-06T21:16:34.405Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1680	2024-12-06T21:16:34.405Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1681	2024-12-06T21:16:34.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1682	2024-12-06T21:16:34.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1683	2024-12-06T21:16:34.408Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1684	2024-12-06T21:16:34.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1685	2024-12-06T21:16:34.410Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1686	2024-12-06T21:16:34.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1687	2024-12-06T21:16:34.413Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1688	2024-12-06T21:16:34.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1689	2024-12-06T21:16:34.415Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1690	2024-12-06T21:16:34.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
1691	2024-12-06T21:16:34.417Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1692	2024-12-06T21:16:34.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1693	2024-12-06T21:16:34.431Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
1694	2024-12-06T21:16:34.431Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.090948ms iteration = 11 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
1695	2024-12-06T21:16:34.439Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1696	2024-12-06T21:16:34.439Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1697	2024-12-06T21:16:34.468Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 62.306112ms iteration = 11 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
1698	2024-12-06T21:16:36.540Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 277 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 9628f1ae-f9a3-4ad3-9500-a1cc631a105d response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
1699	2024-12-06T21:16:44.405Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 12 reason = Timeout
1700	2024-12-06T21:16:44.405Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 12 reason = Timeout
1701	2024-12-06T21:16:44.405Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1702	2024-12-06T21:16:44.405Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1703	2024-12-06T21:16:44.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1704	2024-12-06T21:16:44.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1705	2024-12-06T21:16:44.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1706	2024-12-06T21:16:44.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1707	2024-12-06T21:16:44.410Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1708	2024-12-06T21:16:44.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1709	2024-12-06T21:16:44.412Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1710	2024-12-06T21:16:44.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1711	2024-12-06T21:16:44.413Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1712	2024-12-06T21:16:44.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
1713	2024-12-06T21:16:44.416Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1714	2024-12-06T21:16:44.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1715	2024-12-06T21:16:44.428Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
1716	2024-12-06T21:16:44.428Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.890556ms iteration = 12 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
1717	2024-12-06T21:16:44.435Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1718	2024-12-06T21:16:44.436Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1719	2024-12-06T21:16:44.437Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.968464ms iteration = 12 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
1720	2024-12-06T21:16:46.540Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 420 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 1acbaa3f-8a53-4ae9-8cdd-85253bbbfdef response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
1721	2024-12-06T21:16:51.534Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = 606b13a6-03ff-43b5-8b63-88a26bcc2d37 roles = RoleSet { roles: {} } uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
1722	2024-12-06T21:16:51.534Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = 606b13a6-03ff-43b5-8b63-88a26bcc2d37 resource = Database result = Ok(()) uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
1723	2024-12-06T21:16:51.537Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2995 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = 606b13a6-03ff-43b5-8b63-88a26bcc2d37 response_code = 200 uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
1724	2024-12-06T21:16:51.537Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = ab7d2af1-3756-413a-90c9-bac8b7e0122f roles = RoleSet { roles: {} } uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
1725	2024-12-06T21:16:51.537Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = ab7d2af1-3756-413a-90c9-bac8b7e0122f resource = Database result = Ok(()) uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
1726	2024-12-06T21:16:51.540Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2686 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = ab7d2af1-3756-413a-90c9-bac8b7e0122f response_code = 200 uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
1727	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 5 reason = Timeout
1728	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 5 reason = Timeout
1729	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
1730	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 5 reason = Timeout
1731	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 3 reason = Timeout
1732	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
1733	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1734	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 5 reason = Timeout
1735	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 5 reason = Timeout
1736	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1737	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
1738	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 13 reason = Timeout
1739	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1740	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 3 reason = Timeout
1741	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
1742	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1743	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 6 reason = Timeout
1744	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 5 reason = Timeout
1745	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1746	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
1747	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 5 reason = Timeout
1748	2024-12-06T21:16:54.406Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
1749	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 5 reason = Timeout
1750	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1751	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 5 reason = Timeout
1752	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
1753	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 13 reason = Timeout
1754	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1755	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 3 reason = Timeout
1756	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
1757	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 5 reason = Timeout
1758	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = external_endpoints iteration = 3 reason = Timeout
1759	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1760	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 5 reason = Timeout
1761	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
1762	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 4 reason = Timeout
1763	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 3 reason = Timeout
1764	2024-12-06T21:16:54.406Z	WARN	nexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0
1765	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 12.31\u{b5}s iteration = 4 status = {"enabled":false,"target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0"}
1766	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 5 reason = Timeout
1767	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 5 reason = Timeout
1768	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
1769	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
1770	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
1771	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 5 reason = Timeout
1772	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
1773	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 3 reason = Timeout
1774	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
1775	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1776	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 5 reason = Timeout
1777	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
1778	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 3 reason = Timeout
1779	2024-12-06T21:16:54.406Z	INFO	nexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
1780	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
1781	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 3 reason = Timeout
1782	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 3 reason = Timeout
1783	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 5 reason = Timeout
1784	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 3 reason = Timeout
1785	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1786	2024-12-06T21:16:54.406Z	INFO	nexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-06 21:06:54.406647028 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
1787	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
1788	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
1789	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
1790	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1791	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1792	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1793	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
1794	2024-12-06T21:16:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
1795	2024-12-06T21:16:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1796	2024-12-06T21:16:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1797	2024-12-06T21:16:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1798	2024-12-06T21:16:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
1799	2024-12-06T21:16:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
1800	2024-12-06T21:16:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
1801	2024-12-06T21:16:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1802	2024-12-06T21:16:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1803	2024-12-06T21:16:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
1804	2024-12-06T21:16:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1805	2024-12-06T21:16:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1806	2024-12-06T21:16:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1807	2024-12-06T21:16:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1808	2024-12-06T21:16:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
1809	2024-12-06T21:16:54.407Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4aff1ae3-2d71-40bf-bc46-79a7276da3c2.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4dccdb26-9cf9-40aa-9229-5d35905cf0e1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("79d475dc-44ec-44f4-9e51-ea8a753c39f3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("79d475dc-44ec-44f4-9e51-ea8a753c39f3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 929, tv_nsec: 906055547 } })
1810	2024-12-06T21:16:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
1811	2024-12-06T21:16:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
1812	2024-12-06T21:16:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
1813	2024-12-06T21:16:54.407Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("709da000-e0da-4b29-8160-3112fd080927.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a03854a6-7a78-402b-93b4-65c6d36bf236.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a03854a6-7a78-402b-93b4-65c6d36bf236.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 929, tv_nsec: 906176849 } })
1814	2024-12-06T21:16:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
1815	2024-12-06T21:16:54.407Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
1816	2024-12-06T21:16:54.407Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.258131ms iteration = 3 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
1817	2024-12-06T21:16:54.407Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
1818	2024-12-06T21:16:54.407Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.374212ms iteration = 3 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
1819	2024-12-06T21:16:54.409Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1820	2024-12-06T21:16:54.409Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1821	2024-12-06T21:16:54.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1822	2024-12-06T21:16:54.409Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.498277ms iteration = 5 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
1823	2024-12-06T21:16:54.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
1824	2024-12-06T21:16:54.411Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1825	2024-12-06T21:16:54.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1826	2024-12-06T21:16:54.411Z	DEBG	nexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
1827	2024-12-06T21:16:54.412Z	INFO	nexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
1828	2024-12-06T21:16:54.412Z	DEBG	nexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 5.744565ms iteration = 3 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
1829	2024-12-06T21:16:54.412Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 929, tv_nsec: 911194531 } })
1830	2024-12-06T21:16:54.412Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
1831	2024-12-06T21:16:54.413Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1832	2024-12-06T21:16:54.414Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 8.195744ms iteration = 5 status = {"errors":[],"garbage_collect_requested":[]}
1833	2024-12-06T21:16:54.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1834	2024-12-06T21:16:54.416Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1835	2024-12-06T21:16:54.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1836	2024-12-06T21:16:54.418Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1837	2024-12-06T21:16:54.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1838	2024-12-06T21:16:54.419Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
1839	2024-12-06T21:16:54.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
1840	2024-12-06T21:16:54.419Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 929, tv_nsec: 918709624 } })
1841	2024-12-06T21:16:54.420Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
1842	2024-12-06T21:16:54.433Z	DEBG	nexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 26.409854ms iteration = 3 status = {"errors":[],"found_port_ok":[]}
1843	2024-12-06T21:16:54.434Z	INFO	nexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-06 21:06:54.406647028 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
1844	2024-12-06T21:16:54.434Z	DEBG	nexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 27.626103ms iteration = 3 status = {"expiration":"2024-12-06T21:06:54.406647028Z","pruned":[]}
1845	2024-12-06T21:16:54.434Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1846	2024-12-06T21:16:54.434Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 929, tv_nsec: 933571298 } })
1847	2024-12-06T21:16:54.434Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1848	2024-12-06T21:16:54.435Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
1849	2024-12-06T21:16:54.435Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1850	2024-12-06T21:16:54.440Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1851	2024-12-06T21:16:54.445Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1852	2024-12-06T21:16:54.448Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
1853	2024-12-06T21:16:54.448Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1854	2024-12-06T21:16:54.448Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1855	2024-12-06T21:16:54.454Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
1856	2024-12-06T21:16:54.454Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1857	2024-12-06T21:16:54.459Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1858	2024-12-06T21:16:54.462Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
1859	2024-12-06T21:16:54.462Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1860	2024-12-06T21:16:54.463Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1861	2024-12-06T21:16:54.463Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1862	2024-12-06T21:16:54.464Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1863	2024-12-06T21:16:54.464Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1864	2024-12-06T21:16:54.464Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1865	2024-12-06T21:16:54.466Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
1866	2024-12-06T21:16:54.466Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1867	2024-12-06T21:16:54.467Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
1868	2024-12-06T21:16:54.467Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1869	2024-12-06T21:16:54.467Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 60.658275ms iteration = 13 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
1870	2024-12-06T21:16:54.467Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
1871	2024-12-06T21:16:54.467Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1872	2024-12-06T21:16:54.467Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1873	2024-12-06T21:16:54.467Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
1874	2024-12-06T21:16:54.467Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
1875	2024-12-06T21:16:54.468Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
1876	2024-12-06T21:16:54.468Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
1877	2024-12-06T21:16:54.468Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1878	2024-12-06T21:16:54.469Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1879	2024-12-06T21:16:54.470Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1880	2024-12-06T21:16:54.470Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1881	2024-12-06T21:16:54.470Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1882	2024-12-06T21:16:54.470Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
1883	2024-12-06T21:16:54.470Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
1884	2024-12-06T21:16:54.470Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
1885	2024-12-06T21:16:54.470Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 64.265104ms iteration = 5 status = {"errors":[],"records_set_to_done":[]}
1886	2024-12-06T21:16:54.471Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
1887	2024-12-06T21:16:54.471Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1888	2024-12-06T21:16:54.471Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1889	2024-12-06T21:16:54.471Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1890	2024-12-06T21:16:54.471Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 65.572066ms iteration = 13 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
1891	2024-12-06T21:16:54.472Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1892	2024-12-06T21:16:54.472Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1893	2024-12-06T21:16:54.472Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1894	2024-12-06T21:16:54.472Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1895	2024-12-06T21:16:54.472Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
1896	2024-12-06T21:16:54.472Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1897	2024-12-06T21:16:54.472Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1898	2024-12-06T21:16:54.473Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
1899	2024-12-06T21:16:54.473Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
1900	2024-12-06T21:16:54.474Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
1901	2024-12-06T21:16:54.475Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1902	2024-12-06T21:16:54.475Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
1903	2024-12-06T21:16:54.475Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
1904	2024-12-06T21:16:54.476Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1905	2024-12-06T21:16:54.477Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
1906	2024-12-06T21:16:54.477Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1907	2024-12-06T21:16:54.477Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1908	2024-12-06T21:16:54.478Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1909	2024-12-06T21:16:54.478Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 72.393297ms iteration = 5 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
1910	2024-12-06T21:16:54.479Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
1911	2024-12-06T21:16:54.479Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1912	2024-12-06T21:16:54.480Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1913	2024-12-06T21:16:54.482Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1914	2024-12-06T21:16:54.487Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 80.950807ms iteration = 5 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
1915	2024-12-06T21:16:54.489Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
1916	2024-12-06T21:16:54.489Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
1917	2024-12-06T21:16:54.489Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
1918	2024-12-06T21:16:54.489Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1919	2024-12-06T21:16:54.489Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1920	2024-12-06T21:16:54.491Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1921	2024-12-06T21:16:54.491Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
1922	2024-12-06T21:16:54.491Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1923	2024-12-06T21:16:54.492Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1924	2024-12-06T21:16:54.493Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1925	2024-12-06T21:16:54.493Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1926	2024-12-06T21:16:54.493Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1927	2024-12-06T21:16:54.493Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1928	2024-12-06T21:16:54.495Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1929	2024-12-06T21:16:54.495Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1930	2024-12-06T21:16:54.502Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
1931	2024-12-06T21:16:54.503Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
1932	2024-12-06T21:16:54.515Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
1933	2024-12-06T21:16:54.515Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 109.569597ms iteration = 5 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
1934	2024-12-06T21:16:54.518Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
1935	2024-12-06T21:16:54.518Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
1936	2024-12-06T21:16:54.518Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
1937	2024-12-06T21:16:54.519Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1938	2024-12-06T21:16:54.519Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1939	2024-12-06T21:16:54.520Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1940	2024-12-06T21:16:54.521Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1941	2024-12-06T21:16:54.522Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
1942	2024-12-06T21:16:54.522Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
1943	2024-12-06T21:16:54.522Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
1944	2024-12-06T21:16:54.522Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
1945	2024-12-06T21:16:54.523Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1946	2024-12-06T21:16:54.523Z	DEBG	nexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
1947	2024-12-06T21:16:54.523Z	INFO	nexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
1948	2024-12-06T21:16:54.523Z	DEBG	nexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 117.112331ms iteration = 3 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
1949	2024-12-06T21:16:54.524Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
1950	2024-12-06T21:16:54.524Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1951	2024-12-06T21:16:54.525Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1952	2024-12-06T21:16:54.527Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1953	2024-12-06T21:16:54.527Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
1954	2024-12-06T21:16:54.530Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 123.989264ms iteration = 5 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
1955	2024-12-06T21:16:54.531Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cbc5c1c6-5668-455a-85ef-94f4aebe0f80, lookup_type: ById(cbc5c1c6-5668-455a-85ef-94f4aebe0f80) } result = Ok(())
1956	2024-12-06T21:16:54.531Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1957	2024-12-06T21:16:54.531Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1958	2024-12-06T21:16:54.532Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 126.488785ms iteration = 6 status = {"physical_disks_added":0}
1959	2024-12-06T21:16:54.534Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 127.952779ms iteration = 5 status = 0
1960	2024-12-06T21:16:54.536Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
1961	2024-12-06T21:16:54.536Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1962	2024-12-06T21:16:54.536Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "088c2d2f-d586-4bda-a74b-6ab6db7494c6", "content-length": "26", "date": "Fri, 06 Dec 2024 21:16:54 GMT"} })
1963	2024-12-06T21:16:54.536Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1964	2024-12-06T21:16:54.538Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
1965	2024-12-06T21:16:54.543Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
1966	2024-12-06T21:16:54.543Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 05c53cbc-f66c-491f-af8d-6f291fe21596 dns_zone_name = oxide.test found_so_far = 0 version = 2
1967	2024-12-06T21:16:54.543Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1968	2024-12-06T21:16:54.544Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1969	2024-12-06T21:16:54.545Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
1970	2024-12-06T21:16:54.545Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1971	2024-12-06T21:16:54.550Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1972	2024-12-06T21:16:54.550Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1973	2024-12-06T21:16:54.550Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1974	2024-12-06T21:16:54.552Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 05c53cbc-f66c-491f-af8d-6f291fe21596 dns_zone_name = oxide.test found_so_far = 1 version = 2
1975	2024-12-06T21:16:54.552Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
1976	2024-12-06T21:16:54.552Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-06 21:15:07.045426 UTC dns_group = external generation = 2
1977	2024-12-06T21:16:54.552Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 145.590739ms iteration = 3 status = {"generation":2}
1978	2024-12-06T21:16:54.555Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
1979	2024-12-06T21:16:54.555Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
1980	2024-12-06T21:16:54.555Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 148.860752ms iteration = 5 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
1981	2024-12-06T21:16:54.566Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1982	2024-12-06T21:16:54.568Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1983	2024-12-06T21:16:54.568Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1984	2024-12-06T21:16:54.569Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1985	2024-12-06T21:16:54.569Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1986	2024-12-06T21:16:54.569Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1987	2024-12-06T21:16:54.589Z	DEBG	nexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-06T21:14:50.566199Z, time_modified: 2024-12-06T21:14:50.566199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, db_silo: Silo { identity: SiloIdentity { id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-06T21:15:06.991365Z, time_modified: 2024-12-06T21:15:06.991365Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, db_silo: Silo { identity: SiloIdentity { id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-06T21:15:06.991365Z, time_modified: 2024-12-06T21:15:06.991365Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1" }] }) }
1988	2024-12-06T21:16:54.589Z	DEBG	nexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 182.938921ms iteration = 3 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"683d02e1-6ec8-48bf-98d0-db6c94785f80","tls_certs":["d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1"]}},"default_endpoint":{"silo_id":"683d02e1-6ec8-48bf-98d0-db6c94785f80","tls_certs":["d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
1989	2024-12-06T21:16:54.603Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 zone_address = fd00:1122:3344:101::2
1990	2024-12-06T21:16:54.603Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1991	2024-12-06T21:16:54.603Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "469e3b57-3b53-4026-b6e6-fb7a1ab4c424", "content-length": "26", "date": "Fri, 06 Dec 2024 21:16:54 GMT"} })
1992	2024-12-06T21:16:54.603Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1993	2024-12-06T21:16:54.604Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
1994	2024-12-06T21:16:54.605Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
1995	2024-12-06T21:16:54.605Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 93c3f865-bc00-42df-bef7-20e39342f84f dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
1996	2024-12-06T21:16:54.605Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1997	2024-12-06T21:16:54.605Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1998	2024-12-06T21:16:54.606Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1999	2024-12-06T21:16:54.607Z	WARN	nexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
2000	2024-12-06T21:16:54.607Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 200.844894ms iteration = 3 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
2001	2024-12-06T21:16:54.610Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2002	2024-12-06T21:16:54.610Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2003	2024-12-06T21:16:54.610Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2004	2024-12-06T21:16:54.612Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 93c3f865-bc00-42df-bef7-20e39342f84f dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
2005	2024-12-06T21:16:54.612Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
2006	2024-12-06T21:16:54.612Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-06 21:15:06.025861 UTC dns_group = internal generation = 1
2007	2024-12-06T21:16:54.612Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 205.965769ms iteration = 3 status = {"generation":1}
2008	2024-12-06T21:16:54.614Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
2009	2024-12-06T21:16:54.615Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2010	2024-12-06T21:16:54.615Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "747fdb39-9e42-442d-b767-d78f48a7ee9a", "content-length": "26", "date": "Fri, 06 Dec 2024 21:16:54 GMT"} })
2011	2024-12-06T21:16:54.615Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2012	2024-12-06T21:16:54.617Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
2013	2024-12-06T21:16:54.698Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
2014	2024-12-06T21:16:54.698Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "fda2be72-d945-45ad-8654-6e8bf9d7ed1e", "date": "Fri, 06 Dec 2024 21:16:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2015	2024-12-06T21:16:54.698Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2016	2024-12-06T21:16:54.700Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "19b158cd-33d1-4bd9-8947-2fa1155bfd05", "date": "Fri, 06 Dec 2024 21:16:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2017	2024-12-06T21:16:54.700Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2018	2024-12-06T21:16:54.701Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2019	2024-12-06T21:16:54.701Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 295.146081ms iteration = 5 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
2020	2024-12-06T21:16:54.715Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2021	2024-12-06T21:16:54.717Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2022	2024-12-06T21:16:54.723Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
2023	2024-12-06T21:16:54.762Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
2024	2024-12-06T21:16:54.762Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
2025	2024-12-06T21:16:54.763Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "a7784543-0639-4a57-b177-773c16456436", "content-length": "372", "date": "Fri, 06 Dec 2024 21:16:53 GMT"} })
2026	2024-12-06T21:16:54.763Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2027	2024-12-06T21:16:54.763Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2028	2024-12-06T21:16:54.764Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2029	2024-12-06T21:16:54.766Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
2030	2024-12-06T21:16:54.766Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2031	2024-12-06T21:16:54.766Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2032	2024-12-06T21:16:54.768Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2033	2024-12-06T21:16:54.770Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
2034	2024-12-06T21:16:54.770Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2035	2024-12-06T21:16:54.771Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2036	2024-12-06T21:16:54.813Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
2037	2024-12-06T21:16:54.813Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
2038	2024-12-06T21:16:54.813Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "b90e2696-c9f5-4706-9ce6-a30a1203b583", "date": "Fri, 06 Dec 2024 21:16:53 GMT"} })
2039	2024-12-06T21:16:54.813Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2040	2024-12-06T21:16:54.814Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2041	2024-12-06T21:16:54.816Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2042	2024-12-06T21:16:54.817Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2043	2024-12-06T21:16:54.820Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2044	2024-12-06T21:16:54.820Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2045	2024-12-06T21:16:54.823Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
2046	2024-12-06T21:16:54.823Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
2047	2024-12-06T21:16:54.824Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "8e747b23-9a82-4fb7-b14b-f5c978672062", "date": "Fri, 06 Dec 2024 21:16:53 GMT"} })
2048	2024-12-06T21:16:54.826Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 419.633922ms iteration = 5 status = {}
2049	2024-12-06T21:16:54.844Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 438.188067ms iteration = 5 status = {}
2050	2024-12-06T21:16:54.849Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
2051	2024-12-06T21:16:54.850Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "2ed974e6-41a5-43be-bbb7-a7acfa0197cc", "date": "Fri, 06 Dec 2024 21:16:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
2052	2024-12-06T21:16:54.850Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
2053	2024-12-06T21:16:54.851Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
2054	2024-12-06T21:16:54.851Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "0a17b99b-5e88-4ac3-94dc-b0343c456cb7", "date": "Fri, 06 Dec 2024 21:16:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
2055	2024-12-06T21:16:54.851Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
2056	2024-12-06T21:16:54.852Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "e5055082-7b41-4ffa-b01a-b34b1df99527", "content-length": "1", "date": "Fri, 06 Dec 2024 21:16:53 GMT"} })
2057	2024-12-06T21:16:54.855Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "ce396a74-d03d-45fd-9da9-b8691dc10fea", "date": "Fri, 06 Dec 2024 21:16:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
2058	2024-12-06T21:16:54.855Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
2059	2024-12-06T21:16:54.856Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
2060	2024-12-06T21:16:54.856Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
2061	2024-12-06T21:16:54.857Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
2062	2024-12-06T21:16:54.857Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 451.120609ms iteration = 5 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
2063	2024-12-06T21:16:54.858Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 452.142776ms iteration = 5 status = null
2064	2024-12-06T21:16:54.910Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
2065	2024-12-06T21:16:54.910Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "4271b66e-7bba-4008-a555-54948b14dd32", "content-length": "691", "date": "Fri, 06 Dec 2024 21:16:53 GMT"} })
2066	2024-12-06T21:16:54.910Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
2067	2024-12-06T21:16:54.910Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
2068	2024-12-06T21:16:54.910Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
2069	2024-12-06T21:16:54.912Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 506.0512ms iteration = 5 status = {}
2070	2024-12-06T21:16:54.922Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2071	2024-12-06T21:16:54.922Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2072	2024-12-06T21:16:54.924Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 3d56b7bf-53ed-4147-b45b-9cef154d7338, rack_id: 2d75e4a3-6363-4fd1-935e-989d31fd7d71, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(02859c9b-92d7-4b19-b425-8f266763be54) } rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
2073	2024-12-06T21:16:54.924Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2074	2024-12-06T21:16:54.924Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2075	2024-12-06T21:16:54.934Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
2076	2024-12-06T21:16:54.934Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "655975a8-2bb9-4837-bf87-841c52be5c08", "content-length": "134", "date": "Fri, 06 Dec 2024 21:16:53 GMT"} })
2077	2024-12-06T21:16:54.934Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
2078	2024-12-06T21:16:54.934Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 02859c9b-92d7-4b19-b425-8f266763be54
2079	2024-12-06T21:16:54.935Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
2080	2024-12-06T21:16:54.935Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
2081	2024-12-06T21:16:54.935Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2082	2024-12-06T21:16:54.935Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2083	2024-12-06T21:16:54.935Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2084	2024-12-06T21:16:54.935Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
2085	2024-12-06T21:16:54.935Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2086	2024-12-06T21:16:54.935Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
2087	2024-12-06T21:16:54.935Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
2088	2024-12-06T21:16:54.935Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2089	2024-12-06T21:16:54.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2090	2024-12-06T21:16:54.937Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
2091	2024-12-06T21:16:54.937Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef4b98d4-0239-45a7-8511-b8108868fb17", "content-length": "2", "date": "Fri, 06 Dec 2024 21:16:53 GMT"} })
2092	2024-12-06T21:16:54.937Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
2093	2024-12-06T21:16:54.937Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "a97ecbdf-e487-4638-ac5a-3970e8178d3c", "content-length": "52", "date": "Fri, 06 Dec 2024 21:16:53 GMT"} })
2094	2024-12-06T21:16:54.937Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
2095	2024-12-06T21:16:54.937Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
2096	2024-12-06T21:16:55.029Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "0db3a4c6-a5b9-4caf-8853-ff33e0cd0331", "date": "Fri, 06 Dec 2024 21:16:55 GMT"} })
2097	2024-12-06T21:16:55.029Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch = Switch0
2098	2024-12-06T21:16:55.029Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
2099	2024-12-06T21:16:55.030Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "8eb1d306-3458-41d9-a6d6-4b8b0771afe2", "content-length": "682", "date": "Fri, 06 Dec 2024 21:16:55 GMT"} })
2100	2024-12-06T21:16:55.030Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2101	2024-12-06T21:16:55.030Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2102	2024-12-06T21:16:55.147Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2103	2024-12-06T21:16:55.147Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2104	2024-12-06T21:16:55.151Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2105	2024-12-06T21:16:55.151Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2106	2024-12-06T21:16:55.153Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2107	2024-12-06T21:16:55.153Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2108	2024-12-06T21:16:55.159Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 753.53099ms iteration = 5 status = {}
2109	2024-12-06T21:16:56.540Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 348 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = f2e46bf2-7438-45aa-b50b-47c230fba4fa response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
2110	2024-12-06T21:17:04.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 14 reason = Timeout
2111	2024-12-06T21:17:04.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 14 reason = Timeout
2112	2024-12-06T21:17:04.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2113	2024-12-06T21:17:04.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2114	2024-12-06T21:17:04.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2115	2024-12-06T21:17:04.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2116	2024-12-06T21:17:04.408Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2117	2024-12-06T21:17:04.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2118	2024-12-06T21:17:04.410Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2119	2024-12-06T21:17:04.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2120	2024-12-06T21:17:04.412Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2121	2024-12-06T21:17:04.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2122	2024-12-06T21:17:04.414Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2123	2024-12-06T21:17:04.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
2124	2024-12-06T21:17:04.417Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2125	2024-12-06T21:17:04.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2126	2024-12-06T21:17:04.428Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
2127	2024-12-06T21:17:04.428Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.778747ms iteration = 14 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
2128	2024-12-06T21:17:04.435Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2129	2024-12-06T21:17:04.435Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2130	2024-12-06T21:17:04.436Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 29.839269ms iteration = 14 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
2131	2024-12-06T21:17:06.541Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 310 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 545b6087-0da1-4d26-a806-7708988bf5c7 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
2132	2024-12-06T21:17:14.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 15 reason = Timeout
2133	2024-12-06T21:17:14.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2134	2024-12-06T21:17:14.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 15 reason = Timeout
2135	2024-12-06T21:17:14.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2136	2024-12-06T21:17:14.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2137	2024-12-06T21:17:14.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2138	2024-12-06T21:17:14.408Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2139	2024-12-06T21:17:14.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2140	2024-12-06T21:17:14.408Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2141	2024-12-06T21:17:14.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2142	2024-12-06T21:17:14.411Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2143	2024-12-06T21:17:14.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2144	2024-12-06T21:17:14.413Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2145	2024-12-06T21:17:14.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2146	2024-12-06T21:17:14.415Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2147	2024-12-06T21:17:14.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
2148	2024-12-06T21:17:14.417Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2149	2024-12-06T21:17:14.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2150	2024-12-06T21:17:14.430Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
2151	2024-12-06T21:17:14.430Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.458601ms iteration = 15 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
2152	2024-12-06T21:17:14.437Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.050049ms iteration = 15 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
2153	2024-12-06T21:17:16.540Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 322 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 3d234cce-aeaf-41a4-af46-399c7e6ca125 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
2154	2024-12-06T21:17:21.534Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = 01369b16-e16e-4efe-b862-a7c8628a4192 roles = RoleSet { roles: {} } uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
2155	2024-12-06T21:17:21.534Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = 01369b16-e16e-4efe-b862-a7c8628a4192 resource = Database result = Ok(()) uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
2156	2024-12-06T21:17:21.537Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3109 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = 01369b16-e16e-4efe-b862-a7c8628a4192 response_code = 200 uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
2157	2024-12-06T21:17:21.537Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = 941f0d48-c02d-4ebd-b284-08ecad3e90b9 roles = RoleSet { roles: {} } uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
2158	2024-12-06T21:17:21.537Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = 941f0d48-c02d-4ebd-b284-08ecad3e90b9 resource = Database result = Ok(()) uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
2159	2024-12-06T21:17:21.539Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2465 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = 941f0d48-c02d-4ebd-b284-08ecad3e90b9 response_code = 200 uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
2160	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 6 reason = Timeout
2161	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2162	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 6 reason = Timeout
2163	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
2164	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 6 reason = Timeout
2165	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 6 reason = Timeout
2166	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
2167	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
2168	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2169	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 6 reason = Timeout
2170	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2171	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 6 reason = Timeout
2172	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2173	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 6 reason = Timeout
2174	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
2175	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 6 reason = Timeout
2176	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
2177	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 6 reason = Timeout
2178	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2179	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 6 reason = Timeout
2180	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 6 reason = Timeout
2181	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
2182	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
2183	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 6 reason = Timeout
2184	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 16 reason = Timeout
2185	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2186	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 16 reason = Timeout
2187	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 6 reason = Timeout
2188	2024-12-06T21:17:24.406Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
2189	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 6 reason = Timeout
2190	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2191	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2192	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
2193	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 7 reason = Timeout
2194	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2195	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2196	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2197	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
2198	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
2199	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
2200	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
2201	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2202	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2203	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2204	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
2205	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
2206	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2207	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2208	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
2209	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
2210	2024-12-06T21:17:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2211	2024-12-06T21:17:24.408Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2212	2024-12-06T21:17:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2213	2024-12-06T21:17:24.410Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.24428ms iteration = 6 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
2214	2024-12-06T21:17:24.411Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2215	2024-12-06T21:17:24.411Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
2216	2024-12-06T21:17:24.411Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2217	2024-12-06T21:17:24.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
2218	2024-12-06T21:17:24.411Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2219	2024-12-06T21:17:24.411Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 959, tv_nsec: 910696899 } })
2220	2024-12-06T21:17:24.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2221	2024-12-06T21:17:24.412Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
2222	2024-12-06T21:17:24.412Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 959, tv_nsec: 911187477 } })
2223	2024-12-06T21:17:24.412Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
2224	2024-12-06T21:17:24.414Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2225	2024-12-06T21:17:24.415Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2226	2024-12-06T21:17:24.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2227	2024-12-06T21:17:24.417Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2228	2024-12-06T21:17:24.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2229	2024-12-06T21:17:24.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2230	2024-12-06T21:17:24.418Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2231	2024-12-06T21:17:24.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2232	2024-12-06T21:17:24.420Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 14.05226ms iteration = 16 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
2233	2024-12-06T21:17:24.421Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2234	2024-12-06T21:17:24.421Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2235	2024-12-06T21:17:24.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2236	2024-12-06T21:17:24.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2237	2024-12-06T21:17:24.423Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2238	2024-12-06T21:17:24.427Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
2239	2024-12-06T21:17:24.427Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2240	2024-12-06T21:17:24.427Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2241	2024-12-06T21:17:24.446Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
2242	2024-12-06T21:17:24.446Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 40.338552ms iteration = 16 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
2243	2024-12-06T21:17:24.449Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
2244	2024-12-06T21:17:24.449Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
2245	2024-12-06T21:17:24.450Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2246	2024-12-06T21:17:24.451Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2247	2024-12-06T21:17:24.455Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 48.717159ms iteration = 6 status = {"errors":[],"garbage_collect_requested":[]}
2248	2024-12-06T21:17:24.455Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
2249	2024-12-06T21:17:24.456Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
2250	2024-12-06T21:17:24.456Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2251	2024-12-06T21:17:24.458Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
2252	2024-12-06T21:17:24.458Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
2253	2024-12-06T21:17:24.458Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
2254	2024-12-06T21:17:24.461Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2255	2024-12-06T21:17:24.462Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2256	2024-12-06T21:17:24.462Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 55.948388ms iteration = 6 status = {"errors":[],"records_set_to_done":[]}
2257	2024-12-06T21:17:24.463Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2258	2024-12-06T21:17:24.463Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
2259	2024-12-06T21:17:24.463Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2260	2024-12-06T21:17:24.463Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
2261	2024-12-06T21:17:24.463Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
2262	2024-12-06T21:17:24.463Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2263	2024-12-06T21:17:24.464Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2264	2024-12-06T21:17:24.465Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
2265	2024-12-06T21:17:24.465Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2266	2024-12-06T21:17:24.465Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2267	2024-12-06T21:17:24.466Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2268	2024-12-06T21:17:24.468Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 959, tv_nsec: 967291547 } })
2269	2024-12-06T21:17:24.468Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
2270	2024-12-06T21:17:24.470Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
2271	2024-12-06T21:17:24.470Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2272	2024-12-06T21:17:24.470Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2273	2024-12-06T21:17:24.471Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2274	2024-12-06T21:17:24.471Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2275	2024-12-06T21:17:24.473Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2276	2024-12-06T21:17:24.475Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2277	2024-12-06T21:17:24.475Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2278	2024-12-06T21:17:24.475Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2279	2024-12-06T21:17:24.489Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 83.123554ms iteration = 6 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
2280	2024-12-06T21:17:24.494Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 88.566293ms iteration = 6 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
2281	2024-12-06T21:17:24.495Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
2282	2024-12-06T21:17:24.495Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 89.04126ms iteration = 6 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
2283	2024-12-06T21:17:24.495Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 89.439347ms iteration = 6 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
2284	2024-12-06T21:17:24.495Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2285	2024-12-06T21:17:24.496Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2286	2024-12-06T21:17:24.498Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2287	2024-12-06T21:17:24.498Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2288	2024-12-06T21:17:24.500Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2289	2024-12-06T21:17:24.500Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2290	2024-12-06T21:17:24.500Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2291	2024-12-06T21:17:24.500Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
2292	2024-12-06T21:17:24.500Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 94.312017ms iteration = 6 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
2293	2024-12-06T21:17:24.502Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
2294	2024-12-06T21:17:24.502Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cbc5c1c6-5668-455a-85ef-94f4aebe0f80, lookup_type: ById(cbc5c1c6-5668-455a-85ef-94f4aebe0f80) } result = Ok(())
2295	2024-12-06T21:17:24.502Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2296	2024-12-06T21:17:24.502Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2297	2024-12-06T21:17:24.503Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2298	2024-12-06T21:17:24.503Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2299	2024-12-06T21:17:24.505Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 98.891922ms iteration = 6 status = 0
2300	2024-12-06T21:17:24.505Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
2301	2024-12-06T21:17:24.505Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2302	2024-12-06T21:17:24.505Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b43c091-e7a5-4069-87be-8e4cab2824f9", "content-length": "26", "date": "Fri, 06 Dec 2024 21:17:24 GMT"} })
2303	2024-12-06T21:17:24.505Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2304	2024-12-06T21:17:24.507Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
2305	2024-12-06T21:17:24.514Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
2306	2024-12-06T21:17:24.514Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2307	2024-12-06T21:17:24.514Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e20120af-bd15-4926-94b4-b0423a8ec6b8", "content-length": "26", "date": "Fri, 06 Dec 2024 21:17:24 GMT"} })
2308	2024-12-06T21:17:24.514Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2309	2024-12-06T21:17:24.516Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
2310	2024-12-06T21:17:24.517Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2311	2024-12-06T21:17:24.517Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2312	2024-12-06T21:17:24.525Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2313	2024-12-06T21:17:24.525Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2314	2024-12-06T21:17:24.562Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2315	2024-12-06T21:17:24.563Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
2316	2024-12-06T21:17:24.563Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2317	2024-12-06T21:17:24.564Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2318	2024-12-06T21:17:24.565Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2319	2024-12-06T21:17:24.567Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
2320	2024-12-06T21:17:24.567Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2321	2024-12-06T21:17:24.567Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2322	2024-12-06T21:17:24.573Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 167.269794ms iteration = 7 status = {"physical_disks_added":0}
2323	2024-12-06T21:17:24.595Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 zone_address = fd00:1122:3344:101::2
2324	2024-12-06T21:17:24.595Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2325	2024-12-06T21:17:24.595Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef9c926a-7e00-4911-a125-a6869ce301f9", "content-length": "26", "date": "Fri, 06 Dec 2024 21:17:24 GMT"} })
2326	2024-12-06T21:17:24.595Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2327	2024-12-06T21:17:24.597Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
2328	2024-12-06T21:17:24.610Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 204.300081ms iteration = 6 status = {}
2329	2024-12-06T21:17:24.624Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
2330	2024-12-06T21:17:24.624Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "b9d71f9f-9d83-4c48-8790-e18fc495aaf5", "content-length": "1", "date": "Fri, 06 Dec 2024 21:17:23 GMT"} })
2331	2024-12-06T21:17:24.625Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
2332	2024-12-06T21:17:24.625Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
2333	2024-12-06T21:17:24.745Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
2334	2024-12-06T21:17:24.746Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
2335	2024-12-06T21:17:24.746Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "51dadb26-5d8a-4c54-bc46-4d608961b680", "content-length": "372", "date": "Fri, 06 Dec 2024 21:17:23 GMT"} })
2336	2024-12-06T21:17:24.746Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2337	2024-12-06T21:17:24.746Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2338	2024-12-06T21:17:24.747Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 341.44632ms iteration = 6 status = null
2339	2024-12-06T21:17:24.755Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
2340	2024-12-06T21:17:24.755Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "df08e0e8-f265-4ba6-9d29-46b11e8e68d8", "content-length": "691", "date": "Fri, 06 Dec 2024 21:17:23 GMT"} })
2341	2024-12-06T21:17:24.755Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
2342	2024-12-06T21:17:24.755Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
2343	2024-12-06T21:17:24.755Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
2344	2024-12-06T21:17:24.756Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 350.717813ms iteration = 6 status = {}
2345	2024-12-06T21:17:24.786Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
2346	2024-12-06T21:17:24.786Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
2347	2024-12-06T21:17:24.786Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "e3eee6b6-8296-4b3e-9164-b594b1de3369", "date": "Fri, 06 Dec 2024 21:17:23 GMT"} })
2348	2024-12-06T21:17:24.786Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2349	2024-12-06T21:17:24.787Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2350	2024-12-06T21:17:24.788Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2351	2024-12-06T21:17:24.788Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2352	2024-12-06T21:17:24.791Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2353	2024-12-06T21:17:24.791Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2354	2024-12-06T21:17:24.791Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2355	2024-12-06T21:17:24.791Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2356	2024-12-06T21:17:24.794Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 3d56b7bf-53ed-4147-b45b-9cef154d7338, rack_id: 2d75e4a3-6363-4fd1-935e-989d31fd7d71, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(02859c9b-92d7-4b19-b425-8f266763be54) } rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
2357	2024-12-06T21:17:24.794Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2358	2024-12-06T21:17:24.794Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2359	2024-12-06T21:17:24.795Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
2360	2024-12-06T21:17:24.795Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
2361	2024-12-06T21:17:24.796Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "3fe3a347-210c-40a2-8532-c8df86a7659c", "date": "Fri, 06 Dec 2024 21:17:23 GMT"} })
2362	2024-12-06T21:17:24.797Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 391.315098ms iteration = 6 status = {}
2363	2024-12-06T21:17:24.801Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
2364	2024-12-06T21:17:24.801Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "18f3facd-7974-4e8b-b10f-0f0b07622437", "content-length": "134", "date": "Fri, 06 Dec 2024 21:17:23 GMT"} })
2365	2024-12-06T21:17:24.801Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
2366	2024-12-06T21:17:24.801Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 02859c9b-92d7-4b19-b425-8f266763be54
2367	2024-12-06T21:17:24.801Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
2368	2024-12-06T21:17:24.801Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
2369	2024-12-06T21:17:24.801Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2370	2024-12-06T21:17:24.801Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2371	2024-12-06T21:17:24.801Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2372	2024-12-06T21:17:24.801Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
2373	2024-12-06T21:17:24.802Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2374	2024-12-06T21:17:24.802Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
2375	2024-12-06T21:17:24.802Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
2376	2024-12-06T21:17:24.802Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2377	2024-12-06T21:17:24.802Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2378	2024-12-06T21:17:24.803Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
2379	2024-12-06T21:17:24.803Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "44c63149-da44-48c3-85d5-944371b95b01", "content-length": "2", "date": "Fri, 06 Dec 2024 21:17:23 GMT"} })
2380	2024-12-06T21:17:24.803Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
2381	2024-12-06T21:17:24.803Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "f1c37984-d559-4ca8-92e4-364fb42f4979", "content-length": "52", "date": "Fri, 06 Dec 2024 21:17:23 GMT"} })
2382	2024-12-06T21:17:24.803Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
2383	2024-12-06T21:17:24.803Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
2384	2024-12-06T21:17:24.907Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "547ea9ba-e775-4de6-aa5b-dbf57d3f15eb", "date": "Fri, 06 Dec 2024 21:17:23 GMT"} })
2385	2024-12-06T21:17:24.907Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch = Switch0
2386	2024-12-06T21:17:24.907Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
2387	2024-12-06T21:17:24.908Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "f0ab9b61-07c6-4b7a-bfc8-310a7e01a902", "content-length": "682", "date": "Fri, 06 Dec 2024 21:17:24 GMT"} })
2388	2024-12-06T21:17:24.908Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2389	2024-12-06T21:17:24.908Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2390	2024-12-06T21:17:24.912Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2391	2024-12-06T21:17:24.912Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2392	2024-12-06T21:17:24.914Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2393	2024-12-06T21:17:24.915Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2394	2024-12-06T21:17:24.916Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2395	2024-12-06T21:17:24.916Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2396	2024-12-06T21:17:24.922Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 516.738365ms iteration = 6 status = {}
2397	2024-12-06T21:17:26.540Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 343 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 83cb87a9-bdfa-4c81-84f7-ad6c15ac2f17 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
2398	2024-12-06T21:17:34.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 17 reason = Timeout
2399	2024-12-06T21:17:34.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 17 reason = Timeout
2400	2024-12-06T21:17:34.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2401	2024-12-06T21:17:34.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2402	2024-12-06T21:17:34.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2403	2024-12-06T21:17:34.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2404	2024-12-06T21:17:34.408Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2405	2024-12-06T21:17:34.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2406	2024-12-06T21:17:34.409Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2407	2024-12-06T21:17:34.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2408	2024-12-06T21:17:34.411Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2409	2024-12-06T21:17:34.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2410	2024-12-06T21:17:34.413Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2411	2024-12-06T21:17:34.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2412	2024-12-06T21:17:34.415Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2413	2024-12-06T21:17:34.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
2414	2024-12-06T21:17:34.417Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2415	2024-12-06T21:17:34.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2416	2024-12-06T21:17:34.430Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
2417	2024-12-06T21:17:34.430Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.79391ms iteration = 17 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
2418	2024-12-06T21:17:34.438Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.818012ms iteration = 17 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
2419	2024-12-06T21:17:36.541Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 299 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = ea60da2e-e80d-409d-8769-a2904cb99629 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
2420	2024-12-06T21:17:44.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 18 reason = Timeout
2421	2024-12-06T21:17:44.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 18 reason = Timeout
2422	2024-12-06T21:17:44.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2423	2024-12-06T21:17:44.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2424	2024-12-06T21:17:44.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2425	2024-12-06T21:17:44.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2426	2024-12-06T21:17:44.408Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2427	2024-12-06T21:17:44.408Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2428	2024-12-06T21:17:44.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2429	2024-12-06T21:17:44.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2430	2024-12-06T21:17:44.410Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2431	2024-12-06T21:17:44.410Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.538934ms iteration = 18 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
2432	2024-12-06T21:17:44.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2433	2024-12-06T21:17:44.412Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2434	2024-12-06T21:17:44.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2435	2024-12-06T21:17:44.414Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2436	2024-12-06T21:17:44.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
2437	2024-12-06T21:17:44.416Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2438	2024-12-06T21:17:44.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2439	2024-12-06T21:17:44.428Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
2440	2024-12-06T21:17:44.428Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.913039ms iteration = 18 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
2441	2024-12-06T21:17:46.540Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 308 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = cc330289-127d-44c5-bdc4-1f0dff3451ac response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
2442	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 6 reason = Timeout
2443	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 6 reason = Timeout
2444	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 7 reason = Timeout
2445	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 7 reason = Timeout
2446	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2447	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2448	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 3 reason = Timeout
2449	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 4 reason = Timeout
2450	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
2451	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2452	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2453	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 19 reason = Timeout
2454	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = external_endpoints iteration = 4 reason = Timeout
2455	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2456	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2457	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 7 reason = Timeout
2458	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
2459	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 19 reason = Timeout
2460	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2461	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 7 reason = Timeout
2462	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 7 reason = Timeout
2463	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
2464	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 5 reason = Timeout
2465	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 4 reason = Timeout
2466	2024-12-06T21:17:54.406Z	WARN	nexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0
2467	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 9.791\u{b5}s iteration = 5 status = {"enabled":false,"target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0"}
2468	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
2469	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 7 reason = Timeout
2470	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 4 reason = Timeout
2471	2024-12-06T21:17:54.406Z	INFO	nexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
2472	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
2473	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 7 reason = Timeout
2474	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
2475	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2476	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 4 reason = Timeout
2477	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 7 reason = Timeout
2478	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2479	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 4 reason = Timeout
2480	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 7 reason = Timeout
2481	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
2482	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 4 reason = Timeout
2483	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
2484	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 7 reason = Timeout
2485	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2486	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
2487	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 7 reason = Timeout
2488	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
2489	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2490	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 4 reason = Timeout
2491	2024-12-06T21:17:54.406Z	INFO	nexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-06 21:07:54.406458704 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
2492	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 7 reason = Timeout
2493	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
2494	2024-12-06T21:17:54.406Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
2495	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 4 reason = Timeout
2496	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 7 reason = Timeout
2497	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2498	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
2499	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 7 reason = Timeout
2500	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
2501	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
2502	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 8 reason = Timeout
2503	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 4 reason = Timeout
2504	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2505	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
2506	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2507	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2508	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2509	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
2510	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2511	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
2512	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
2513	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2514	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2515	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
2516	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2517	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2518	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
2519	2024-12-06T21:17:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
2520	2024-12-06T21:17:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2521	2024-12-06T21:17:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2522	2024-12-06T21:17:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2523	2024-12-06T21:17:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
2524	2024-12-06T21:17:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
2525	2024-12-06T21:17:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
2526	2024-12-06T21:17:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
2527	2024-12-06T21:17:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
2528	2024-12-06T21:17:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
2529	2024-12-06T21:17:54.407Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4aff1ae3-2d71-40bf-bc46-79a7276da3c2.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4dccdb26-9cf9-40aa-9229-5d35905cf0e1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("79d475dc-44ec-44f4-9e51-ea8a753c39f3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("79d475dc-44ec-44f4-9e51-ea8a753c39f3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 989, tv_nsec: 905878224 } })
2530	2024-12-06T21:17:54.407Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("709da000-e0da-4b29-8160-3112fd080927.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a03854a6-7a78-402b-93b4-65c6d36bf236.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a03854a6-7a78-402b-93b4-65c6d36bf236.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 989, tv_nsec: 905874214 } })
2531	2024-12-06T21:17:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
2532	2024-12-06T21:17:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2533	2024-12-06T21:17:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2534	2024-12-06T21:17:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
2535	2024-12-06T21:17:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
2536	2024-12-06T21:17:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
2537	2024-12-06T21:17:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2538	2024-12-06T21:17:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
2539	2024-12-06T21:17:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2540	2024-12-06T21:17:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
2541	2024-12-06T21:17:54.407Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
2542	2024-12-06T21:17:54.407Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.277441ms iteration = 4 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
2543	2024-12-06T21:17:54.407Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
2544	2024-12-06T21:17:54.407Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.278001ms iteration = 4 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
2545	2024-12-06T21:17:54.409Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2546	2024-12-06T21:17:54.409Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2547	2024-12-06T21:17:54.409Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.620799ms iteration = 7 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
2548	2024-12-06T21:17:54.410Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2549	2024-12-06T21:17:54.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
2550	2024-12-06T21:17:54.411Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2551	2024-12-06T21:17:54.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2552	2024-12-06T21:17:54.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
2553	2024-12-06T21:17:54.412Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2554	2024-12-06T21:17:54.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2555	2024-12-06T21:17:54.412Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
2556	2024-12-06T21:17:54.412Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2557	2024-12-06T21:17:54.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2558	2024-12-06T21:17:54.414Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2559	2024-12-06T21:17:54.414Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 989, tv_nsec: 913362227 } })
2560	2024-12-06T21:17:54.414Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
2561	2024-12-06T21:17:54.414Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2562	2024-12-06T21:17:54.415Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2563	2024-12-06T21:17:54.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2564	2024-12-06T21:17:54.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2565	2024-12-06T21:17:54.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
2566	2024-12-06T21:17:54.416Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2567	2024-12-06T21:17:54.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
2568	2024-12-06T21:17:54.417Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2569	2024-12-06T21:17:54.417Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2570	2024-12-06T21:17:54.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2571	2024-12-06T21:17:54.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
2572	2024-12-06T21:17:54.419Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
2573	2024-12-06T21:17:54.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
2574	2024-12-06T21:17:54.419Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2575	2024-12-06T21:17:54.421Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 14.603139ms iteration = 7 status = {"errors":[],"records_set_to_done":[]}
2576	2024-12-06T21:17:54.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2577	2024-12-06T21:17:54.421Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2578	2024-12-06T21:17:54.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2579	2024-12-06T21:17:54.426Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 20.011588ms iteration = 7 status = {"errors":[],"garbage_collect_requested":[]}
2580	2024-12-06T21:17:54.427Z	INFO	nexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-06 21:07:54.406458704 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
2581	2024-12-06T21:17:54.427Z	DEBG	nexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 20.952174ms iteration = 4 status = {"expiration":"2024-12-06T21:07:54.406458704Z","pruned":[]}
2582	2024-12-06T21:17:54.428Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2583	2024-12-06T21:17:54.428Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 989, tv_nsec: 927377907 } })
2584	2024-12-06T21:17:54.428Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
2585	2024-12-06T21:17:54.429Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
2586	2024-12-06T21:17:54.430Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2587	2024-12-06T21:17:54.430Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2588	2024-12-06T21:17:54.430Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2589	2024-12-06T21:17:54.430Z	DEBG	nexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 23.830871ms iteration = 4 status = {"errors":[],"found_port_ok":[]}
2590	2024-12-06T21:17:54.431Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 989, tv_nsec: 930314105 } })
2591	2024-12-06T21:17:54.431Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
2592	2024-12-06T21:17:54.432Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
2593	2024-12-06T21:17:54.432Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2594	2024-12-06T21:17:54.433Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
2595	2024-12-06T21:17:54.434Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2596	2024-12-06T21:17:54.434Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2597	2024-12-06T21:17:54.435Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 28.815443ms iteration = 19 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
2598	2024-12-06T21:17:54.435Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2599	2024-12-06T21:17:54.435Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2600	2024-12-06T21:17:54.435Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2601	2024-12-06T21:17:54.438Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2602	2024-12-06T21:17:54.439Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
2603	2024-12-06T21:17:54.439Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
2604	2024-12-06T21:17:54.439Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
2605	2024-12-06T21:17:54.439Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2606	2024-12-06T21:17:54.439Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2607	2024-12-06T21:17:54.439Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2608	2024-12-06T21:17:54.439Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2609	2024-12-06T21:17:54.440Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2610	2024-12-06T21:17:54.442Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
2611	2024-12-06T21:17:54.442Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2612	2024-12-06T21:17:54.442Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2613	2024-12-06T21:17:54.446Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2614	2024-12-06T21:17:54.448Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2615	2024-12-06T21:17:54.448Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2616	2024-12-06T21:17:54.450Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2617	2024-12-06T21:17:54.450Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2618	2024-12-06T21:17:54.450Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2619	2024-12-06T21:17:54.451Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2620	2024-12-06T21:17:54.451Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2621	2024-12-06T21:17:54.451Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2622	2024-12-06T21:17:54.453Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2623	2024-12-06T21:17:54.453Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
2624	2024-12-06T21:17:54.453Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2625	2024-12-06T21:17:54.453Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2626	2024-12-06T21:17:54.455Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2627	2024-12-06T21:17:54.455Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2628	2024-12-06T21:17:54.455Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2629	2024-12-06T21:17:54.462Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2630	2024-12-06T21:17:54.464Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2631	2024-12-06T21:17:54.464Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2632	2024-12-06T21:17:54.465Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2633	2024-12-06T21:17:54.465Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
2634	2024-12-06T21:17:54.465Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
2635	2024-12-06T21:17:54.465Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
2636	2024-12-06T21:17:54.466Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
2637	2024-12-06T21:17:54.466Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2638	2024-12-06T21:17:54.466Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
2639	2024-12-06T21:17:54.468Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2640	2024-12-06T21:17:54.468Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2641	2024-12-06T21:17:54.469Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2642	2024-12-06T21:17:54.469Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
2643	2024-12-06T21:17:54.469Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2644	2024-12-06T21:17:54.469Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
2645	2024-12-06T21:17:54.469Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2646	2024-12-06T21:17:54.469Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
2647	2024-12-06T21:17:54.469Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
2648	2024-12-06T21:17:54.469Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2649	2024-12-06T21:17:54.473Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
2650	2024-12-06T21:17:54.473Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 66.896667ms iteration = 7 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
2651	2024-12-06T21:17:54.473Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2652	2024-12-06T21:17:54.476Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
2653	2024-12-06T21:17:54.476Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2654	2024-12-06T21:17:54.476Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
2655	2024-12-06T21:17:54.476Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2656	2024-12-06T21:17:54.477Z	DEBG	nexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
2657	2024-12-06T21:17:54.477Z	INFO	nexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
2658	2024-12-06T21:17:54.477Z	DEBG	nexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 70.503497ms iteration = 4 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
2659	2024-12-06T21:17:54.478Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2660	2024-12-06T21:17:54.478Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2661	2024-12-06T21:17:54.478Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
2662	2024-12-06T21:17:54.478Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2663	2024-12-06T21:17:54.478Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
2664	2024-12-06T21:17:54.480Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
2665	2024-12-06T21:17:54.481Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
2666	2024-12-06T21:17:54.489Z	DEBG	nexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
2667	2024-12-06T21:17:54.489Z	INFO	nexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
2668	2024-12-06T21:17:54.489Z	DEBG	nexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 83.233895ms iteration = 4 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
2669	2024-12-06T21:17:54.489Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2670	2024-12-06T21:17:54.492Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2671	2024-12-06T21:17:54.497Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
2672	2024-12-06T21:17:54.497Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
2673	2024-12-06T21:17:54.505Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
2674	2024-12-06T21:17:54.505Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 93c3f865-bc00-42df-bef7-20e39342f84f dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
2675	2024-12-06T21:17:54.505Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2676	2024-12-06T21:17:54.506Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2677	2024-12-06T21:17:54.509Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2678	2024-12-06T21:17:54.509Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2679	2024-12-06T21:17:54.509Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2680	2024-12-06T21:17:54.509Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2681	2024-12-06T21:17:54.522Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
2682	2024-12-06T21:17:54.522Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
2683	2024-12-06T21:17:54.522Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
2684	2024-12-06T21:17:54.522Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2685	2024-12-06T21:17:54.522Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2686	2024-12-06T21:17:54.531Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
2687	2024-12-06T21:17:54.555Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 149.086565ms iteration = 8 status = {"physical_disks_added":0}
2688	2024-12-06T21:17:54.574Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
2689	2024-12-06T21:17:54.574Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2690	2024-12-06T21:17:54.574Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2fc530f0-b910-4ea7-9891-b9cc8ae9cda7", "content-length": "26", "date": "Fri, 06 Dec 2024 21:17:54 GMT"} })
2691	2024-12-06T21:17:54.574Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2692	2024-12-06T21:17:54.576Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 zone_address = fd00:1122:3344:101::2
2693	2024-12-06T21:17:54.576Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2694	2024-12-06T21:17:54.577Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce40fe4b-3daf-4635-aa07-289bad635806", "content-length": "26", "date": "Fri, 06 Dec 2024 21:17:54 GMT"} })
2695	2024-12-06T21:17:54.577Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2696	2024-12-06T21:17:54.577Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
2697	2024-12-06T21:17:54.579Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
2698	2024-12-06T21:17:54.619Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
2699	2024-12-06T21:17:54.684Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
2700	2024-12-06T21:17:54.684Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
2701	2024-12-06T21:17:54.685Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "4796bb1d-16ab-4cbb-8ebd-55077f4d65d4", "content-length": "372", "date": "Fri, 06 Dec 2024 21:17:54 GMT"} })
2702	2024-12-06T21:17:54.685Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2703	2024-12-06T21:17:54.685Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2704	2024-12-06T21:17:54.687Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2705	2024-12-06T21:17:54.687Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
2706	2024-12-06T21:17:54.687Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2707	2024-12-06T21:17:54.687Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2708	2024-12-06T21:17:54.687Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
2709	2024-12-06T21:17:54.691Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
2710	2024-12-06T21:17:54.691Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2711	2024-12-06T21:17:54.691Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2712	2024-12-06T21:17:54.694Z	DEBG	nexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 288.163756ms iteration = 3 status = {"nsuccess":5}
2713	2024-12-06T21:17:54.694Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2714	2024-12-06T21:17:54.694Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 288.175227ms iteration = 7 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
2715	2024-12-06T21:17:54.694Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2716	2024-12-06T21:17:54.696Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2717	2024-12-06T21:17:54.700Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2718	2024-12-06T21:17:54.700Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2719	2024-12-06T21:17:54.700Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2720	2024-12-06T21:17:54.701Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2721	2024-12-06T21:17:54.705Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2722	2024-12-06T21:17:54.705Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2723	2024-12-06T21:17:54.706Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2724	2024-12-06T21:17:54.709Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 93c3f865-bc00-42df-bef7-20e39342f84f dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
2725	2024-12-06T21:17:54.709Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
2726	2024-12-06T21:17:54.709Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-06 21:15:06.025861 UTC dns_group = internal generation = 1
2727	2024-12-06T21:17:54.710Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 303.812393ms iteration = 4 status = {"generation":1}
2728	2024-12-06T21:17:54.718Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
2729	2024-12-06T21:17:54.725Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
2730	2024-12-06T21:17:54.725Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2731	2024-12-06T21:17:54.725Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c30082e-672d-48bb-adcf-2d9b6c47c441", "content-length": "26", "date": "Fri, 06 Dec 2024 21:17:54 GMT"} })
2732	2024-12-06T21:17:54.725Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2733	2024-12-06T21:17:54.727Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
2734	2024-12-06T21:17:54.731Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
2735	2024-12-06T21:17:54.731Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 325.56116ms iteration = 19 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
2736	2024-12-06T21:17:54.743Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 336.929626ms iteration = 7 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
2737	2024-12-06T21:17:54.748Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 342.370226ms iteration = 7 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
2738	2024-12-06T21:17:54.750Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2739	2024-12-06T21:17:54.751Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2740	2024-12-06T21:17:54.752Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
2741	2024-12-06T21:17:54.752Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 345.566219ms iteration = 7 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
2742	2024-12-06T21:17:54.753Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
2743	2024-12-06T21:17:54.753Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2744	2024-12-06T21:17:54.753Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2745	2024-12-06T21:17:54.758Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
2746	2024-12-06T21:17:54.758Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 05c53cbc-f66c-491f-af8d-6f291fe21596 dns_zone_name = oxide.test found_so_far = 0 version = 2
2747	2024-12-06T21:17:54.758Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2748	2024-12-06T21:17:54.758Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2749	2024-12-06T21:17:54.759Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2750	2024-12-06T21:17:54.762Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2751	2024-12-06T21:17:54.762Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2752	2024-12-06T21:17:54.762Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2753	2024-12-06T21:17:54.762Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
2754	2024-12-06T21:17:54.763Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "425023df-40fe-4e6f-926c-02549717a787", "content-length": "1", "date": "Fri, 06 Dec 2024 21:17:54 GMT"} })
2755	2024-12-06T21:17:54.764Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 05c53cbc-f66c-491f-af8d-6f291fe21596 dns_zone_name = oxide.test found_so_far = 1 version = 2
2756	2024-12-06T21:17:54.764Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
2757	2024-12-06T21:17:54.764Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-06 21:15:07.045426 UTC dns_group = external generation = 2
2758	2024-12-06T21:17:54.764Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 358.552681ms iteration = 4 status = {"generation":2}
2759	2024-12-06T21:17:54.765Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
2760	2024-12-06T21:17:54.765Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2761	2024-12-06T21:17:54.765Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
2762	2024-12-06T21:17:54.769Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cbc5c1c6-5668-455a-85ef-94f4aebe0f80, lookup_type: ById(cbc5c1c6-5668-455a-85ef-94f4aebe0f80) } result = Ok(())
2763	2024-12-06T21:17:54.769Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2764	2024-12-06T21:17:54.769Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2765	2024-12-06T21:17:54.772Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 365.896262ms iteration = 7 status = null
2766	2024-12-06T21:17:54.772Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
2767	2024-12-06T21:17:54.772Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 366.111995ms iteration = 7 status = 0
2768	2024-12-06T21:17:54.772Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
2769	2024-12-06T21:17:54.772Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "92f5f4b8-99f6-46c8-a43e-4da71f8991a7", "date": "Fri, 06 Dec 2024 21:17:54 GMT"} })
2770	2024-12-06T21:17:54.772Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2771	2024-12-06T21:17:54.772Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2772	2024-12-06T21:17:54.775Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2773	2024-12-06T21:17:54.775Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2774	2024-12-06T21:17:54.779Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2775	2024-12-06T21:17:54.779Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2776	2024-12-06T21:17:54.783Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
2777	2024-12-06T21:17:54.783Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
2778	2024-12-06T21:17:54.783Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "f61e14a2-affb-4221-ae9e-d6844c0bbe31", "date": "Fri, 06 Dec 2024 21:17:54 GMT"} })
2779	2024-12-06T21:17:54.785Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 378.655131ms iteration = 7 status = {}
2780	2024-12-06T21:17:54.803Z	DEBG	nexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-06T21:14:50.566199Z, time_modified: 2024-12-06T21:14:50.566199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, db_silo: Silo { identity: SiloIdentity { id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-06T21:15:06.991365Z, time_modified: 2024-12-06T21:15:06.991365Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, db_silo: Silo { identity: SiloIdentity { id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-06T21:15:06.991365Z, time_modified: 2024-12-06T21:15:06.991365Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1" }] }) }
2781	2024-12-06T21:17:54.803Z	DEBG	nexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 397.050753ms iteration = 4 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"683d02e1-6ec8-48bf-98d0-db6c94785f80","tls_certs":["d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1"]}},"default_endpoint":{"silo_id":"683d02e1-6ec8-48bf-98d0-db6c94785f80","tls_certs":["d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
2782	2024-12-06T21:17:54.812Z	WARN	nexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
2783	2024-12-06T21:17:54.812Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 406.358185ms iteration = 4 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
2784	2024-12-06T21:17:54.816Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
2785	2024-12-06T21:17:54.816Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "4ade0298-b499-4edd-93d1-8405f5bea8ba", "date": "Fri, 06 Dec 2024 21:17:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2786	2024-12-06T21:17:54.816Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2787	2024-12-06T21:17:54.818Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "a81be358-3fb5-418d-9825-7d55728bf896", "date": "Fri, 06 Dec 2024 21:17:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2788	2024-12-06T21:17:54.818Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2789	2024-12-06T21:17:54.820Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2790	2024-12-06T21:17:54.820Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 414.048562ms iteration = 6 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
2791	2024-12-06T21:17:54.837Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 430.827596ms iteration = 7 status = {}
2792	2024-12-06T21:17:54.837Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
2793	2024-12-06T21:17:54.837Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "60662f09-6670-4295-8b9b-3ede05c0eda3", "date": "Fri, 06 Dec 2024 21:17:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
2794	2024-12-06T21:17:54.837Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
2795	2024-12-06T21:17:54.839Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "64075a0d-7195-4809-8187-54eed3bf37d9", "date": "Fri, 06 Dec 2024 21:17:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
2796	2024-12-06T21:17:54.839Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
2797	2024-12-06T21:17:54.840Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2798	2024-12-06T21:17:54.840Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2799	2024-12-06T21:17:54.841Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "95b4c710-40fa-4eac-a238-b12ac7070606", "date": "Fri, 06 Dec 2024 21:17:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
2800	2024-12-06T21:17:54.841Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
2801	2024-12-06T21:17:54.842Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2802	2024-12-06T21:17:54.842Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
2803	2024-12-06T21:17:54.843Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 436.919537ms iteration = 6 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
2804	2024-12-06T21:17:54.843Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
2805	2024-12-06T21:17:54.844Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2806	2024-12-06T21:17:54.844Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2807	2024-12-06T21:17:54.845Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2808	2024-12-06T21:17:54.847Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
2809	2024-12-06T21:17:54.847Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2810	2024-12-06T21:17:54.847Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2811	2024-12-06T21:17:54.885Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2812	2024-12-06T21:17:54.886Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2813	2024-12-06T21:17:54.893Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 3d56b7bf-53ed-4147-b45b-9cef154d7338, rack_id: 2d75e4a3-6363-4fd1-935e-989d31fd7d71, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(02859c9b-92d7-4b19-b425-8f266763be54) } rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
2814	2024-12-06T21:17:54.893Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2815	2024-12-06T21:17:54.893Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2816	2024-12-06T21:17:54.907Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
2817	2024-12-06T21:17:54.908Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "3317fdcc-7c73-4ba0-84f4-2031a80a157f", "content-length": "134", "date": "Fri, 06 Dec 2024 21:17:54 GMT"} })
2818	2024-12-06T21:17:54.908Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
2819	2024-12-06T21:17:54.908Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 02859c9b-92d7-4b19-b425-8f266763be54
2820	2024-12-06T21:17:54.908Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
2821	2024-12-06T21:17:54.908Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
2822	2024-12-06T21:17:54.908Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2823	2024-12-06T21:17:54.908Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2824	2024-12-06T21:17:54.908Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2825	2024-12-06T21:17:54.908Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
2826	2024-12-06T21:17:54.908Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2827	2024-12-06T21:17:54.908Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
2828	2024-12-06T21:17:54.909Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
2829	2024-12-06T21:17:54.909Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2830	2024-12-06T21:17:54.909Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2831	2024-12-06T21:17:54.910Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
2832	2024-12-06T21:17:54.910Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "70d7d75f-7803-4f83-8daf-2eb4ec5034eb", "content-length": "2", "date": "Fri, 06 Dec 2024 21:17:54 GMT"} })
2833	2024-12-06T21:17:54.911Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
2834	2024-12-06T21:17:54.911Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "0366c7d8-53c7-426c-9c81-043304f88a36", "content-length": "52", "date": "Fri, 06 Dec 2024 21:17:54 GMT"} })
2835	2024-12-06T21:17:54.911Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
2836	2024-12-06T21:17:54.911Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
2837	2024-12-06T21:17:54.948Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
2838	2024-12-06T21:17:55.011Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "ab564190-2895-4093-8f1b-5e187da34ad4", "date": "Fri, 06 Dec 2024 21:17:54 GMT"} })
2839	2024-12-06T21:17:55.011Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch = Switch0
2840	2024-12-06T21:17:55.011Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
2841	2024-12-06T21:17:55.011Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "f289f302-e5b6-4f4b-8244-c535d9ccd4a1", "content-length": "682", "date": "Fri, 06 Dec 2024 21:17:55 GMT"} })
2842	2024-12-06T21:17:55.011Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2843	2024-12-06T21:17:55.011Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2844	2024-12-06T21:17:55.012Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "bce78aa6-17dd-4fc9-a668-4623e10dd17d", "content-length": "691", "date": "Fri, 06 Dec 2024 21:17:54 GMT"} })
2845	2024-12-06T21:17:55.012Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
2846	2024-12-06T21:17:55.012Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
2847	2024-12-06T21:17:55.012Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
2848	2024-12-06T21:17:55.013Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 607.441443ms iteration = 7 status = {}
2849	2024-12-06T21:17:55.015Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2850	2024-12-06T21:17:55.015Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2851	2024-12-06T21:17:55.018Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2852	2024-12-06T21:17:55.019Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2853	2024-12-06T21:17:55.020Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2854	2024-12-06T21:17:55.021Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2855	2024-12-06T21:17:55.027Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 621.729327ms iteration = 7 status = {}
2856	2024-12-06T21:17:56.540Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 288 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = a65a583f-cb38-4e5a-8bdf-1c5910f84b98 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
2857	2024-12-06T21:18:04.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 20 reason = Timeout
2858	2024-12-06T21:18:04.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 20 reason = Timeout
2859	2024-12-06T21:18:04.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2860	2024-12-06T21:18:04.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2861	2024-12-06T21:18:04.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2862	2024-12-06T21:18:04.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2863	2024-12-06T21:18:04.408Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2864	2024-12-06T21:18:04.409Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2865	2024-12-06T21:18:04.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2866	2024-12-06T21:18:04.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2867	2024-12-06T21:18:04.411Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2868	2024-12-06T21:18:04.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2869	2024-12-06T21:18:04.412Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2870	2024-12-06T21:18:04.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2871	2024-12-06T21:18:04.414Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2872	2024-12-06T21:18:04.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
2873	2024-12-06T21:18:04.416Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2874	2024-12-06T21:18:04.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2875	2024-12-06T21:18:04.429Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
2876	2024-12-06T21:18:04.429Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.090399ms iteration = 20 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
2877	2024-12-06T21:18:04.438Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.498983ms iteration = 20 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
2878	2024-12-06T21:18:06.535Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = f00d5e00-33f9-415d-9ce7-26c0d416b331 roles = RoleSet { roles: {} } uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
2879	2024-12-06T21:18:06.535Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = f00d5e00-33f9-415d-9ce7-26c0d416b331 resource = Database result = Ok(()) uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
2880	2024-12-06T21:18:06.538Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3530 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = f00d5e00-33f9-415d-9ce7-26c0d416b331 response_code = 200 uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
2881	2024-12-06T21:18:06.538Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = 8bc19d6d-082c-48d3-805e-fad7a3e9c6cb roles = RoleSet { roles: {} } uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
2882	2024-12-06T21:18:06.539Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = 8bc19d6d-082c-48d3-805e-fad7a3e9c6cb resource = Database result = Ok(()) uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
2883	2024-12-06T21:18:06.540Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1781 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = 8bc19d6d-082c-48d3-805e-fad7a3e9c6cb response_code = 200 uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
2884	2024-12-06T21:18:06.541Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 386 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = aeac1e0e-46e8-4490-b146-6c8033e1e10f response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
2885	2024-12-06T21:18:14.405Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 21 reason = Timeout
2886	2024-12-06T21:18:14.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 21 reason = Timeout
2887	2024-12-06T21:18:14.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2888	2024-12-06T21:18:14.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2889	2024-12-06T21:18:14.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2890	2024-12-06T21:18:14.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2891	2024-12-06T21:18:14.408Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2892	2024-12-06T21:18:14.409Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2893	2024-12-06T21:18:14.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2894	2024-12-06T21:18:14.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2895	2024-12-06T21:18:14.410Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2896	2024-12-06T21:18:14.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2897	2024-12-06T21:18:14.412Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2898	2024-12-06T21:18:14.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2899	2024-12-06T21:18:14.414Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2900	2024-12-06T21:18:14.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
2901	2024-12-06T21:18:14.416Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2902	2024-12-06T21:18:14.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2903	2024-12-06T21:18:14.428Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
2904	2024-12-06T21:18:14.428Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.679332ms iteration = 21 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
2905	2024-12-06T21:18:14.441Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 35.064995ms iteration = 21 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
2906	2024-12-06T21:18:16.541Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 416 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = aa485a7e-5137-4f31-af36-b7887e0f39f5 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
2907	2024-12-06T21:18:21.534Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = 63113dba-6f5c-42d2-bd59-8e34f3be8b19 roles = RoleSet { roles: {} } uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
2908	2024-12-06T21:18:21.534Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = 63113dba-6f5c-42d2-bd59-8e34f3be8b19 resource = Database result = Ok(()) uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
2909	2024-12-06T21:18:21.537Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3132 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = 63113dba-6f5c-42d2-bd59-8e34f3be8b19 response_code = 200 uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
2910	2024-12-06T21:18:21.537Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = 37616539-73e4-40a0-b712-4d6cc0b6b915 roles = RoleSet { roles: {} } uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
2911	2024-12-06T21:18:21.537Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = 37616539-73e4-40a0-b712-4d6cc0b6b915 resource = Database result = Ok(()) uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
2912	2024-12-06T21:18:21.539Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2613 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = 37616539-73e4-40a0-b712-4d6cc0b6b915 response_code = 200 uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
2913	2024-12-06T21:18:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 8 reason = Timeout
2914	2024-12-06T21:18:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2915	2024-12-06T21:18:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 8 reason = Timeout
2916	2024-12-06T21:18:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 8 reason = Timeout
2917	2024-12-06T21:18:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
2918	2024-12-06T21:18:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2919	2024-12-06T21:18:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 8 reason = Timeout
2920	2024-12-06T21:18:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 8 reason = Timeout
2921	2024-12-06T21:18:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
2922	2024-12-06T21:18:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2923	2024-12-06T21:18:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 22 reason = Timeout
2924	2024-12-06T21:18:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2925	2024-12-06T21:18:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 8 reason = Timeout
2926	2024-12-06T21:18:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 22 reason = Timeout
2927	2024-12-06T21:18:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 8 reason = Timeout
2928	2024-12-06T21:18:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
2929	2024-12-06T21:18:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2930	2024-12-06T21:18:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 8 reason = Timeout
2931	2024-12-06T21:18:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 8 reason = Timeout
2932	2024-12-06T21:18:24.406Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
2933	2024-12-06T21:18:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
2934	2024-12-06T21:18:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2935	2024-12-06T21:18:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 8 reason = Timeout
2936	2024-12-06T21:18:24.406Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
2937	2024-12-06T21:18:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2938	2024-12-06T21:18:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 8 reason = Timeout
2939	2024-12-06T21:18:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
2940	2024-12-06T21:18:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 8 reason = Timeout
2941	2024-12-06T21:18:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 8 reason = Timeout
2942	2024-12-06T21:18:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
2943	2024-12-06T21:18:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2944	2024-12-06T21:18:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 9 reason = Timeout
2945	2024-12-06T21:18:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2946	2024-12-06T21:18:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 8 reason = Timeout
2947	2024-12-06T21:18:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
2948	2024-12-06T21:18:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2949	2024-12-06T21:18:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
2950	2024-12-06T21:18:24.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2951	2024-12-06T21:18:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2952	2024-12-06T21:18:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
2953	2024-12-06T21:18:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
2954	2024-12-06T21:18:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2955	2024-12-06T21:18:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
2956	2024-12-06T21:18:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2957	2024-12-06T21:18:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2958	2024-12-06T21:18:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
2959	2024-12-06T21:18:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
2960	2024-12-06T21:18:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
2961	2024-12-06T21:18:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2962	2024-12-06T21:18:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
2963	2024-12-06T21:18:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2964	2024-12-06T21:18:24.408Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2965	2024-12-06T21:18:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2966	2024-12-06T21:18:24.409Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.918518ms iteration = 8 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
2967	2024-12-06T21:18:24.409Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2968	2024-12-06T21:18:24.410Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2969	2024-12-06T21:18:24.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2970	2024-12-06T21:18:24.411Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1019, tv_nsec: 909878314 } })
2971	2024-12-06T21:18:24.411Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2972	2024-12-06T21:18:24.411Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1019, tv_nsec: 910108137 } })
2973	2024-12-06T21:18:24.411Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
2974	2024-12-06T21:18:24.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
2975	2024-12-06T21:18:24.411Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2976	2024-12-06T21:18:24.411Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
2977	2024-12-06T21:18:24.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2978	2024-12-06T21:18:24.413Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2979	2024-12-06T21:18:24.413Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2980	2024-12-06T21:18:24.413Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2981	2024-12-06T21:18:24.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
2982	2024-12-06T21:18:24.414Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2983	2024-12-06T21:18:24.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2984	2024-12-06T21:18:24.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
2985	2024-12-06T21:18:24.415Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
2986	2024-12-06T21:18:24.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
2987	2024-12-06T21:18:24.416Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
2988	2024-12-06T21:18:24.416Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2989	2024-12-06T21:18:24.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2990	2024-12-06T21:18:24.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2991	2024-12-06T21:18:24.417Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2992	2024-12-06T21:18:24.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2993	2024-12-06T21:18:24.418Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2994	2024-12-06T21:18:24.419Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 12.797059ms iteration = 8 status = {"errors":[],"garbage_collect_requested":[]}
2995	2024-12-06T21:18:24.420Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2996	2024-12-06T21:18:24.420Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2997	2024-12-06T21:18:24.420Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2998	2024-12-06T21:18:24.422Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1019, tv_nsec: 921552395 } })
2999	2024-12-06T21:18:24.422Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3000	2024-12-06T21:18:24.423Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3001	2024-12-06T21:18:24.423Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
3002	2024-12-06T21:18:24.424Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3003	2024-12-06T21:18:24.424Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3004	2024-12-06T21:18:24.424Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3005	2024-12-06T21:18:24.425Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3006	2024-12-06T21:18:24.426Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3007	2024-12-06T21:18:24.428Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
3008	2024-12-06T21:18:24.428Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3009	2024-12-06T21:18:24.428Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3010	2024-12-06T21:18:24.460Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 53.939975ms iteration = 22 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
3011	2024-12-06T21:18:24.460Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
3012	2024-12-06T21:18:24.460Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
3013	2024-12-06T21:18:24.460Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
3014	2024-12-06T21:18:24.460Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
3015	2024-12-06T21:18:24.461Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
3016	2024-12-06T21:18:24.461Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
3017	2024-12-06T21:18:24.461Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
3018	2024-12-06T21:18:24.461Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3019	2024-12-06T21:18:24.462Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
3020	2024-12-06T21:18:24.462Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
3021	2024-12-06T21:18:24.462Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3022	2024-12-06T21:18:24.462Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
3023	2024-12-06T21:18:24.462Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 56.522507ms iteration = 8 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
3024	2024-12-06T21:18:24.463Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 56.771081ms iteration = 8 status = {"errors":[],"records_set_to_done":[]}
3025	2024-12-06T21:18:24.463Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
3026	2024-12-06T21:18:24.463Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
3027	2024-12-06T21:18:24.463Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
3028	2024-12-06T21:18:24.463Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
3029	2024-12-06T21:18:24.464Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3030	2024-12-06T21:18:24.465Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3031	2024-12-06T21:18:24.466Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3032	2024-12-06T21:18:24.467Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3033	2024-12-06T21:18:24.469Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cbc5c1c6-5668-455a-85ef-94f4aebe0f80, lookup_type: ById(cbc5c1c6-5668-455a-85ef-94f4aebe0f80) } result = Ok(())
3034	2024-12-06T21:18:24.469Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3035	2024-12-06T21:18:24.469Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3036	2024-12-06T21:18:24.469Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
3037	2024-12-06T21:18:24.469Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3038	2024-12-06T21:18:24.469Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3039	2024-12-06T21:18:24.471Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 65.057027ms iteration = 8 status = 0
3040	2024-12-06T21:18:24.471Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3041	2024-12-06T21:18:24.474Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
3042	2024-12-06T21:18:24.474Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 67.586658ms iteration = 8 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
3043	2024-12-06T21:18:24.474Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
3044	2024-12-06T21:18:24.474Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3045	2024-12-06T21:18:24.474Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3046	2024-12-06T21:18:24.489Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
3047	2024-12-06T21:18:24.490Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 83.440758ms iteration = 22 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
3048	2024-12-06T21:18:24.490Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
3049	2024-12-06T21:18:24.490Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3050	2024-12-06T21:18:24.491Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f3f71b55-9a6b-4dba-9d26-4453e824aa86", "content-length": "26", "date": "Fri, 06 Dec 2024 21:18:24 GMT"} })
3051	2024-12-06T21:18:24.491Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3052	2024-12-06T21:18:24.492Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
3053	2024-12-06T21:18:24.492Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3054	2024-12-06T21:18:24.492Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "35a3c9cd-92e9-48bb-86b0-25cb07698a34", "content-length": "26", "date": "Fri, 06 Dec 2024 21:18:24 GMT"} })
3055	2024-12-06T21:18:24.492Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3056	2024-12-06T21:18:24.492Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3057	2024-12-06T21:18:24.494Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3058	2024-12-06T21:18:24.495Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 88.585873ms iteration = 8 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
3059	2024-12-06T21:18:24.495Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 89.221373ms iteration = 8 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
3060	2024-12-06T21:18:24.496Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
3061	2024-12-06T21:18:24.496Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
3062	2024-12-06T21:18:24.496Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 89.443517ms iteration = 8 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
3063	2024-12-06T21:18:24.502Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 zone_address = fd00:1122:3344:101::2
3064	2024-12-06T21:18:24.502Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3065	2024-12-06T21:18:24.502Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f1456207-bb61-443a-aea4-f2188090dc04", "content-length": "26", "date": "Fri, 06 Dec 2024 21:18:24 GMT"} })
3066	2024-12-06T21:18:24.503Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3067	2024-12-06T21:18:24.504Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
3068	2024-12-06T21:18:24.535Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3069	2024-12-06T21:18:24.535Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3070	2024-12-06T21:18:24.537Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 131.260144ms iteration = 9 status = {"physical_disks_added":0}
3071	2024-12-06T21:18:24.566Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
3072	2024-12-06T21:18:24.566Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
3073	2024-12-06T21:18:24.566Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "212c9223-ed28-4e38-9447-6a60a1f0236b", "content-length": "372", "date": "Fri, 06 Dec 2024 21:18:23 GMT"} })
3074	2024-12-06T21:18:24.566Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3075	2024-12-06T21:18:24.567Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3076	2024-12-06T21:18:24.569Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3077	2024-12-06T21:18:24.569Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3078	2024-12-06T21:18:24.590Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
3079	2024-12-06T21:18:24.591Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "a222430d-7f61-4f24-be13-10b78b76a298", "content-length": "1", "date": "Fri, 06 Dec 2024 21:18:23 GMT"} })
3080	2024-12-06T21:18:24.592Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3081	2024-12-06T21:18:24.593Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3082	2024-12-06T21:18:24.593Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 186.764873ms iteration = 8 status = {}
3083	2024-12-06T21:18:24.595Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 188.598583ms iteration = 8 status = null
3084	2024-12-06T21:18:24.598Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3085	2024-12-06T21:18:24.600Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
3086	2024-12-06T21:18:24.600Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3087	2024-12-06T21:18:24.600Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3088	2024-12-06T21:18:24.601Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3089	2024-12-06T21:18:24.603Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
3090	2024-12-06T21:18:24.603Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3091	2024-12-06T21:18:24.603Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3092	2024-12-06T21:18:24.608Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
3093	2024-12-06T21:18:24.608Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
3094	2024-12-06T21:18:24.608Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "f2eee972-6c16-4951-b74c-caf9bfea7009", "date": "Fri, 06 Dec 2024 21:18:23 GMT"} })
3095	2024-12-06T21:18:24.608Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3096	2024-12-06T21:18:24.609Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3097	2024-12-06T21:18:24.611Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3098	2024-12-06T21:18:24.611Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3099	2024-12-06T21:18:24.615Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3100	2024-12-06T21:18:24.615Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3101	2024-12-06T21:18:24.619Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
3102	2024-12-06T21:18:24.619Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
3103	2024-12-06T21:18:24.619Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "ed533598-9670-4f38-95b7-a6cae92cd168", "date": "Fri, 06 Dec 2024 21:18:23 GMT"} })
3104	2024-12-06T21:18:24.621Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 214.805723ms iteration = 8 status = {}
3105	2024-12-06T21:18:24.674Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3106	2024-12-06T21:18:24.674Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3107	2024-12-06T21:18:24.676Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 3d56b7bf-53ed-4147-b45b-9cef154d7338, rack_id: 2d75e4a3-6363-4fd1-935e-989d31fd7d71, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(02859c9b-92d7-4b19-b425-8f266763be54) } rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
3108	2024-12-06T21:18:24.676Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3109	2024-12-06T21:18:24.677Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3110	2024-12-06T21:18:24.689Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
3111	2024-12-06T21:18:24.689Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "6170581a-f214-4532-abbd-9f054360efb7", "content-length": "134", "date": "Fri, 06 Dec 2024 21:18:23 GMT"} })
3112	2024-12-06T21:18:24.689Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
3113	2024-12-06T21:18:24.689Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 02859c9b-92d7-4b19-b425-8f266763be54
3114	2024-12-06T21:18:24.690Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
3115	2024-12-06T21:18:24.690Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
3116	2024-12-06T21:18:24.690Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3117	2024-12-06T21:18:24.690Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3118	2024-12-06T21:18:24.690Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3119	2024-12-06T21:18:24.690Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
3120	2024-12-06T21:18:24.690Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3121	2024-12-06T21:18:24.690Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
3122	2024-12-06T21:18:24.690Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
3123	2024-12-06T21:18:24.690Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3124	2024-12-06T21:18:24.691Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3125	2024-12-06T21:18:24.695Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
3126	2024-12-06T21:18:24.696Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "b8e782c3-403f-4272-a56a-cb91cb69e4f2", "content-length": "2", "date": "Fri, 06 Dec 2024 21:18:23 GMT"} })
3127	2024-12-06T21:18:24.696Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
3128	2024-12-06T21:18:24.696Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "d3b6f993-3048-4148-b89e-644d53b5f554", "content-length": "52", "date": "Fri, 06 Dec 2024 21:18:23 GMT"} })
3129	2024-12-06T21:18:24.696Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
3130	2024-12-06T21:18:24.696Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
3131	2024-12-06T21:18:24.725Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
3132	2024-12-06T21:18:24.796Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "fe435d5e-7fc0-4957-b2a1-a9b625024eff", "date": "Fri, 06 Dec 2024 21:18:24 GMT"} })
3133	2024-12-06T21:18:24.796Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch = Switch0
3134	2024-12-06T21:18:24.796Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
3135	2024-12-06T21:18:24.796Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "46678e36-dc73-43d6-9699-ef57a50b4436", "content-length": "691", "date": "Fri, 06 Dec 2024 21:18:24 GMT"} })
3136	2024-12-06T21:18:24.796Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
3137	2024-12-06T21:18:24.796Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
3138	2024-12-06T21:18:24.797Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
3139	2024-12-06T21:18:24.797Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "44b89d6b-1421-45d0-838f-6a3a581dd52a", "content-length": "682", "date": "Fri, 06 Dec 2024 21:18:24 GMT"} })
3140	2024-12-06T21:18:24.797Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3141	2024-12-06T21:18:24.797Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3142	2024-12-06T21:18:24.798Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 392.316215ms iteration = 8 status = {}
3143	2024-12-06T21:18:24.801Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3144	2024-12-06T21:18:24.801Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3145	2024-12-06T21:18:24.805Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3146	2024-12-06T21:18:24.805Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3147	2024-12-06T21:18:24.807Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3148	2024-12-06T21:18:24.807Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3149	2024-12-06T21:18:24.814Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 407.929081ms iteration = 8 status = {}
3150	2024-12-06T21:18:24.910Z	DEBG	nexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
3151	2024-12-06T21:18:24.910Z	DEBG	nexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
3152	2024-12-06T21:18:24.941Z	DEBG	nexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
3153	2024-12-06T21:18:24.941Z	INFO	nexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:56555
3154	2024-12-06T21:18:24.941Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:56555 req_id = dd815e41-9983-435d-95f6-39515e0c5b59 roles = RoleSet { roles: {} } uri = /metrics/producers
3155	2024-12-06T21:18:24.942Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:56555 req_id = dd815e41-9983-435d-95f6-39515e0c5b59 resource = Database result = Ok(()) uri = /metrics/producers
3156	2024-12-06T21:18:24.978Z	INFO	nexus: assigned collector to new producer collector_id = 9d509eff-9671-46fb-8274-6ec9019a9d9e file = nexus/src/app/oximeter.rs:76 producer_id = e8725c8c-13ab-4916-bf16-bb7f832014d5
3157	2024-12-06T21:18:24.978Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 36860 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:56555 req_id = dd815e41-9983-435d-95f6-39515e0c5b59 response_code = 201 uri = /metrics/producers
3158	2024-12-06T21:18:24.978Z	DEBG	nexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "dd815e41-9983-435d-95f6-39515e0c5b59", "content-length": "41", "date": "Fri, 06 Dec 2024 21:18:24 GMT"} })
3159	2024-12-06T21:18:24.980Z	DEBG	nexus (producer-registration-task): registered with nexus successfully
3160	2024-12-06T21:18:24.980Z	DEBG	nexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
3161	2024-12-06T21:18:26.541Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 298 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = ce417626-57c3-470e-964e-be93742f60c2 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
3162	2024-12-06T21:18:34.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 23 reason = Timeout
3163	2024-12-06T21:18:34.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 23 reason = Timeout
3164	2024-12-06T21:18:34.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3165	2024-12-06T21:18:34.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3166	2024-12-06T21:18:34.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3167	2024-12-06T21:18:34.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3168	2024-12-06T21:18:34.439Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3169	2024-12-06T21:18:34.439Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3170	2024-12-06T21:18:34.440Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3171	2024-12-06T21:18:34.441Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 35.074495ms iteration = 23 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
3172	2024-12-06T21:18:34.442Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3173	2024-12-06T21:18:34.442Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3174	2024-12-06T21:18:34.442Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3175	2024-12-06T21:18:34.445Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3176	2024-12-06T21:18:34.445Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3177	2024-12-06T21:18:34.446Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3178	2024-12-06T21:18:34.449Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
3179	2024-12-06T21:18:34.449Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3180	2024-12-06T21:18:34.449Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3181	2024-12-06T21:18:34.460Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
3182	2024-12-06T21:18:34.460Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 53.617269ms iteration = 23 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
3183	2024-12-06T21:18:36.534Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = e94cb211-bd77-48ab-a0f8-233fe690d21c roles = RoleSet { roles: {} } uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
3184	2024-12-06T21:18:36.534Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = e94cb211-bd77-48ab-a0f8-233fe690d21c resource = Database result = Ok(()) uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
3185	2024-12-06T21:18:36.537Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2691 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = e94cb211-bd77-48ab-a0f8-233fe690d21c response_code = 200 uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
3186	2024-12-06T21:18:36.537Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = d4faeafa-068b-4963-8075-b9bd54838c34 roles = RoleSet { roles: {} } uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
3187	2024-12-06T21:18:36.537Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = d4faeafa-068b-4963-8075-b9bd54838c34 resource = Database result = Ok(()) uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
3188	2024-12-06T21:18:36.539Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1889 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = d4faeafa-068b-4963-8075-b9bd54838c34 response_code = 200 uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
3189	2024-12-06T21:18:36.540Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 251 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 1822bdc5-d5dd-4c08-9b00-e42ca615f164 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
3190	2024-12-06T21:18:44.405Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 24 reason = Timeout
3191	2024-12-06T21:18:44.405Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 24 reason = Timeout
3192	2024-12-06T21:18:44.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3193	2024-12-06T21:18:44.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3194	2024-12-06T21:18:44.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3195	2024-12-06T21:18:44.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3196	2024-12-06T21:18:44.408Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3197	2024-12-06T21:18:44.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3198	2024-12-06T21:18:44.411Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3199	2024-12-06T21:18:44.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3200	2024-12-06T21:18:44.412Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3201	2024-12-06T21:18:44.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3202	2024-12-06T21:18:44.414Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3203	2024-12-06T21:18:44.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
3204	2024-12-06T21:18:44.417Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3205	2024-12-06T21:18:44.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3206	2024-12-06T21:18:44.426Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
3207	2024-12-06T21:18:44.426Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.783381ms iteration = 24 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
3208	2024-12-06T21:18:44.436Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3209	2024-12-06T21:18:44.436Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3210	2024-12-06T21:18:44.464Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 59.003297ms iteration = 24 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
3211	2024-12-06T21:18:46.540Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 278 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 48421331-72a8-45a0-9caa-e11635e3f9c5 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
3212	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 7 reason = Timeout
3213	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 5 reason = Timeout
3214	2024-12-06T21:18:54.406Z	INFO	nexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-06 21:08:54.406274986 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
3215	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
3216	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 6 reason = Timeout
3217	2024-12-06T21:18:54.406Z	WARN	nexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0
3218	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 19.19\u{b5}s iteration = 6 status = {"enabled":false,"target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0"}
3219	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 5 reason = Timeout
3220	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 25 reason = Timeout
3221	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 9 reason = Timeout
3222	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3223	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 9 reason = Timeout
3224	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
3225	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3226	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 5 reason = Timeout
3227	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 5 reason = Timeout
3228	2024-12-06T21:18:54.406Z	INFO	nexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
3229	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
3230	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 9 reason = Timeout
3231	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
3232	2024-12-06T21:18:54.406Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
3233	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 9 reason = Timeout
3234	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3235	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
3236	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 9 reason = Timeout
3237	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 9 reason = Timeout
3238	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
3239	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = external_endpoints iteration = 5 reason = Timeout
3240	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 5 reason = Timeout
3241	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3242	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
3243	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 10 reason = Timeout
3244	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 9 reason = Timeout
3245	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3246	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3247	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 9 reason = Timeout
3248	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3249	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 9 reason = Timeout
3250	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
3251	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 9 reason = Timeout
3252	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
3253	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
3254	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 7 reason = Timeout
3255	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 5 reason = Timeout
3256	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
3257	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 9 reason = Timeout
3258	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 5 reason = Timeout
3259	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3260	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
3261	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3262	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 5 reason = Timeout
3263	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
3264	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3265	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 9 reason = Timeout
3266	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
3267	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 5 reason = Timeout
3268	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 9 reason = Timeout
3269	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
3270	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 9 reason = Timeout
3271	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
3272	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 25 reason = Timeout
3273	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3274	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
3275	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
3276	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3277	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3278	2024-12-06T21:18:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
3279	2024-12-06T21:18:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
3280	2024-12-06T21:18:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
3281	2024-12-06T21:18:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3282	2024-12-06T21:18:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
3283	2024-12-06T21:18:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
3284	2024-12-06T21:18:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3285	2024-12-06T21:18:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3286	2024-12-06T21:18:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3287	2024-12-06T21:18:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
3288	2024-12-06T21:18:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3289	2024-12-06T21:18:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
3290	2024-12-06T21:18:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
3291	2024-12-06T21:18:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3292	2024-12-06T21:18:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3293	2024-12-06T21:18:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3294	2024-12-06T21:18:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
3295	2024-12-06T21:18:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
3296	2024-12-06T21:18:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
3297	2024-12-06T21:18:54.407Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("709da000-e0da-4b29-8160-3112fd080927.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a03854a6-7a78-402b-93b4-65c6d36bf236.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a03854a6-7a78-402b-93b4-65c6d36bf236.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1049, tv_nsec: 905871541 } })
3298	2024-12-06T21:18:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3299	2024-12-06T21:18:54.407Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4aff1ae3-2d71-40bf-bc46-79a7276da3c2.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4dccdb26-9cf9-40aa-9229-5d35905cf0e1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("79d475dc-44ec-44f4-9e51-ea8a753c39f3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("79d475dc-44ec-44f4-9e51-ea8a753c39f3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1049, tv_nsec: 905919372 } })
3300	2024-12-06T21:18:54.407Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
3301	2024-12-06T21:18:54.407Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.003626ms iteration = 5 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
3302	2024-12-06T21:18:54.407Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
3303	2024-12-06T21:18:54.407Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.453523ms iteration = 5 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
3304	2024-12-06T21:18:54.408Z	INFO	nexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-06 21:08:54.406274986 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
3305	2024-12-06T21:18:54.408Z	DEBG	nexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 2.573062ms iteration = 5 status = {"expiration":"2024-12-06T21:08:54.406274986Z","pruned":[]}
3306	2024-12-06T21:18:54.409Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3307	2024-12-06T21:18:54.410Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.201549ms iteration = 9 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
3308	2024-12-06T21:18:54.410Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3309	2024-12-06T21:18:54.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
3310	2024-12-06T21:18:54.411Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
3311	2024-12-06T21:18:54.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
3312	2024-12-06T21:18:54.412Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1049, tv_nsec: 911508973 } })
3313	2024-12-06T21:18:54.413Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3314	2024-12-06T21:18:54.413Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
3315	2024-12-06T21:18:54.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
3316	2024-12-06T21:18:54.413Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3317	2024-12-06T21:18:54.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3318	2024-12-06T21:18:54.414Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
3319	2024-12-06T21:18:54.414Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3320	2024-12-06T21:18:54.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3321	2024-12-06T21:18:54.416Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3322	2024-12-06T21:18:54.416Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3323	2024-12-06T21:18:54.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3324	2024-12-06T21:18:54.417Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3325	2024-12-06T21:18:54.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
3326	2024-12-06T21:18:54.418Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3327	2024-12-06T21:18:54.418Z	DEBG	nexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 11.892465ms iteration = 5 status = {"errors":[],"found_port_ok":[]}
3328	2024-12-06T21:18:54.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3329	2024-12-06T21:18:54.418Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
3330	2024-12-06T21:18:54.418Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3331	2024-12-06T21:18:54.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3332	2024-12-06T21:18:54.419Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3333	2024-12-06T21:18:54.420Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3334	2024-12-06T21:18:54.421Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1049, tv_nsec: 920524161 } })
3335	2024-12-06T21:18:54.422Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
3336	2024-12-06T21:18:54.424Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3337	2024-12-06T21:18:54.425Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1049, tv_nsec: 924012738 } })
3338	2024-12-06T21:18:54.425Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
3339	2024-12-06T21:18:54.426Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3340	2024-12-06T21:18:54.426Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
3341	2024-12-06T21:18:54.426Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3342	2024-12-06T21:18:54.426Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3343	2024-12-06T21:18:54.426Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3344	2024-12-06T21:18:54.426Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3345	2024-12-06T21:18:54.426Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3346	2024-12-06T21:18:54.428Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3347	2024-12-06T21:18:54.431Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
3348	2024-12-06T21:18:54.431Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3349	2024-12-06T21:18:54.431Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3350	2024-12-06T21:18:54.433Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3351	2024-12-06T21:18:54.437Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
3352	2024-12-06T21:18:54.437Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3353	2024-12-06T21:18:54.438Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3354	2024-12-06T21:18:54.456Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
3355	2024-12-06T21:18:54.456Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
3356	2024-12-06T21:18:54.456Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
3357	2024-12-06T21:18:54.456Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
3358	2024-12-06T21:18:54.464Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3359	2024-12-06T21:18:54.464Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
3360	2024-12-06T21:18:54.465Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
3361	2024-12-06T21:18:54.467Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3362	2024-12-06T21:18:54.469Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3363	2024-12-06T21:18:54.469Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3364	2024-12-06T21:18:54.469Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
3365	2024-12-06T21:18:54.469Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3366	2024-12-06T21:18:54.469Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3367	2024-12-06T21:18:54.470Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
3368	2024-12-06T21:18:54.470Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
3369	2024-12-06T21:18:54.470Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
3370	2024-12-06T21:18:54.470Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 64.267224ms iteration = 9 status = {"errors":[],"records_set_to_done":[]}
3371	2024-12-06T21:18:54.470Z	DEBG	nexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
3372	2024-12-06T21:18:54.470Z	INFO	nexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
3373	2024-12-06T21:18:54.470Z	DEBG	nexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 64.477787ms iteration = 5 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
3374	2024-12-06T21:18:54.471Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 65.114928ms iteration = 9 status = {"errors":[],"garbage_collect_requested":[]}
3375	2024-12-06T21:18:54.471Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 65.451894ms iteration = 9 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
3376	2024-12-06T21:18:54.473Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
3377	2024-12-06T21:18:54.473Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
3378	2024-12-06T21:18:54.473Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3379	2024-12-06T21:18:54.473Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3380	2024-12-06T21:18:54.474Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
3381	2024-12-06T21:18:54.474Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 67.67754ms iteration = 9 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
3382	2024-12-06T21:18:54.474Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3383	2024-12-06T21:18:54.475Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
3384	2024-12-06T21:18:54.475Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
3385	2024-12-06T21:18:54.476Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3386	2024-12-06T21:18:54.476Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3387	2024-12-06T21:18:54.476Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3388	2024-12-06T21:18:54.479Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3389	2024-12-06T21:18:54.479Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3390	2024-12-06T21:18:54.479Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3391	2024-12-06T21:18:54.481Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3392	2024-12-06T21:18:54.481Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3393	2024-12-06T21:18:54.482Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
3394	2024-12-06T21:18:54.484Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
3395	2024-12-06T21:18:54.484Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3396	2024-12-06T21:18:54.484Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3397	2024-12-06T21:18:54.484Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
3398	2024-12-06T21:18:54.497Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3399	2024-12-06T21:18:54.498Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3400	2024-12-06T21:18:54.498Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3401	2024-12-06T21:18:54.499Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
3402	2024-12-06T21:18:54.499Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 92.612189ms iteration = 25 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
3403	2024-12-06T21:18:54.499Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3404	2024-12-06T21:18:54.499Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3405	2024-12-06T21:18:54.501Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3406	2024-12-06T21:18:54.501Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
3407	2024-12-06T21:18:54.501Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3408	2024-12-06T21:18:54.501Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3409	2024-12-06T21:18:54.502Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3410	2024-12-06T21:18:54.502Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
3411	2024-12-06T21:18:54.502Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
3412	2024-12-06T21:18:54.505Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3413	2024-12-06T21:18:54.507Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
3414	2024-12-06T21:18:54.507Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
3415	2024-12-06T21:18:54.507Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cbc5c1c6-5668-455a-85ef-94f4aebe0f80, lookup_type: ById(cbc5c1c6-5668-455a-85ef-94f4aebe0f80) } result = Ok(())
3416	2024-12-06T21:18:54.507Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3417	2024-12-06T21:18:54.508Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3418	2024-12-06T21:18:54.510Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 103.957266ms iteration = 9 status = 0
3419	2024-12-06T21:18:54.514Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 108.077372ms iteration = 25 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
3420	2024-12-06T21:18:54.514Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
3421	2024-12-06T21:18:54.514Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
3422	2024-12-06T21:18:54.514Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
3423	2024-12-06T21:18:54.514Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3424	2024-12-06T21:18:54.514Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 108.443488ms iteration = 9 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
3425	2024-12-06T21:18:54.515Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3426	2024-12-06T21:18:54.515Z	DEBG	nexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
3427	2024-12-06T21:18:54.515Z	INFO	nexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
3428	2024-12-06T21:18:54.515Z	DEBG	nexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 109.127659ms iteration = 5 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
3429	2024-12-06T21:18:54.516Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3430	2024-12-06T21:18:54.519Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
3431	2024-12-06T21:18:54.519Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3432	2024-12-06T21:18:54.519Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
3433	2024-12-06T21:18:54.519Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
3434	2024-12-06T21:18:54.519Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
3435	2024-12-06T21:18:54.519Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3436	2024-12-06T21:18:54.519Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
3437	2024-12-06T21:18:54.519Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
3438	2024-12-06T21:18:54.519Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
3439	2024-12-06T21:18:54.519Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3440	2024-12-06T21:18:54.519Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3441	2024-12-06T21:18:54.519Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
3442	2024-12-06T21:18:54.521Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3443	2024-12-06T21:18:54.524Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
3444	2024-12-06T21:18:54.524Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3445	2024-12-06T21:18:54.524Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3446	2024-12-06T21:18:54.541Z	DEBG	nexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-06T21:14:50.566199Z, time_modified: 2024-12-06T21:14:50.566199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, db_silo: Silo { identity: SiloIdentity { id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-06T21:15:06.991365Z, time_modified: 2024-12-06T21:15:06.991365Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, db_silo: Silo { identity: SiloIdentity { id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-06T21:15:06.991365Z, time_modified: 2024-12-06T21:15:06.991365Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1" }] }) }
3447	2024-12-06T21:18:54.541Z	DEBG	nexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 135.189307ms iteration = 5 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"683d02e1-6ec8-48bf-98d0-db6c94785f80","tls_certs":["d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1"]}},"default_endpoint":{"silo_id":"683d02e1-6ec8-48bf-98d0-db6c94785f80","tls_certs":["d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
3448	2024-12-06T21:18:54.548Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 142.220003ms iteration = 9 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
3449	2024-12-06T21:18:54.554Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3450	2024-12-06T21:18:54.555Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3451	2024-12-06T21:18:54.559Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
3452	2024-12-06T21:18:54.559Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 05c53cbc-f66c-491f-af8d-6f291fe21596 dns_zone_name = oxide.test found_so_far = 0 version = 2
3453	2024-12-06T21:18:54.559Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3454	2024-12-06T21:18:54.559Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3455	2024-12-06T21:18:54.559Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
3456	2024-12-06T21:18:54.559Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 93c3f865-bc00-42df-bef7-20e39342f84f dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
3457	2024-12-06T21:18:54.559Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3458	2024-12-06T21:18:54.560Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3459	2024-12-06T21:18:54.560Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
3460	2024-12-06T21:18:54.561Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3461	2024-12-06T21:18:54.563Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
3462	2024-12-06T21:18:54.563Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3463	2024-12-06T21:18:54.563Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3464	2024-12-06T21:18:54.565Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 05c53cbc-f66c-491f-af8d-6f291fe21596 dns_zone_name = oxide.test found_so_far = 1 version = 2
3465	2024-12-06T21:18:54.565Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
3466	2024-12-06T21:18:54.565Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-06 21:15:07.045426 UTC dns_group = external generation = 2
3467	2024-12-06T21:18:54.565Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 159.206931ms iteration = 5 status = {"generation":2}
3468	2024-12-06T21:18:54.565Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
3469	2024-12-06T21:18:54.565Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
3470	2024-12-06T21:18:54.565Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 159.304143ms iteration = 9 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
3471	2024-12-06T21:18:54.566Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 159.75264ms iteration = 10 status = {"physical_disks_added":0}
3472	2024-12-06T21:18:54.576Z	WARN	nexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
3473	2024-12-06T21:18:54.576Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 170.108ms iteration = 5 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
3474	2024-12-06T21:18:54.587Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3475	2024-12-06T21:18:54.590Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
3476	2024-12-06T21:18:54.590Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3477	2024-12-06T21:18:54.590Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3478	2024-12-06T21:18:54.591Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3479	2024-12-06T21:18:54.591Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3480	2024-12-06T21:18:54.592Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3481	2024-12-06T21:18:54.593Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 93c3f865-bc00-42df-bef7-20e39342f84f dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
3482	2024-12-06T21:18:54.593Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
3483	2024-12-06T21:18:54.593Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-06 21:15:06.025861 UTC dns_group = internal generation = 1
3484	2024-12-06T21:18:54.593Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 186.766303ms iteration = 5 status = {"generation":1}
3485	2024-12-06T21:18:54.594Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
3486	2024-12-06T21:18:54.594Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3487	2024-12-06T21:18:54.594Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3488	2024-12-06T21:18:54.595Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3489	2024-12-06T21:18:54.597Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
3490	2024-12-06T21:18:54.597Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3491	2024-12-06T21:18:54.597Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3492	2024-12-06T21:18:54.689Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
3493	2024-12-06T21:18:54.720Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
3494	2024-12-06T21:18:54.721Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3495	2024-12-06T21:18:54.721Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ccae804c-3500-4fa8-970c-1221ecc52727", "content-length": "26", "date": "Fri, 06 Dec 2024 21:18:54 GMT"} })
3496	2024-12-06T21:18:54.721Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3497	2024-12-06T21:18:54.723Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3498	2024-12-06T21:18:54.737Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
3499	2024-12-06T21:18:54.737Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3500	2024-12-06T21:18:54.737Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8d105c6f-2373-4537-893c-ba3924956c78", "content-length": "26", "date": "Fri, 06 Dec 2024 21:18:54 GMT"} })
3501	2024-12-06T21:18:54.737Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3502	2024-12-06T21:18:54.741Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3503	2024-12-06T21:18:54.746Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 zone_address = fd00:1122:3344:101::2
3504	2024-12-06T21:18:54.746Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3505	2024-12-06T21:18:54.747Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5601472a-8911-4a13-8049-3bcfb22a68c1", "content-length": "26", "date": "Fri, 06 Dec 2024 21:18:54 GMT"} })
3506	2024-12-06T21:18:54.747Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3507	2024-12-06T21:18:54.748Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
3508	2024-12-06T21:18:54.778Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
3509	2024-12-06T21:18:54.778Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
3510	2024-12-06T21:18:54.779Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f0dce52-d1e1-4811-9f8a-95865d383612", "content-length": "372", "date": "Fri, 06 Dec 2024 21:18:53 GMT"} })
3511	2024-12-06T21:18:54.779Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3512	2024-12-06T21:18:54.779Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
3513	2024-12-06T21:18:54.779Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3514	2024-12-06T21:18:54.779Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "ab4a8322-bb72-47a7-b01f-daba18a7a695", "date": "Fri, 06 Dec 2024 21:18:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
3515	2024-12-06T21:18:54.779Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
3516	2024-12-06T21:18:54.781Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "7db2bd59-5a12-4a72-aa5e-ad16e072b8bd", "date": "Fri, 06 Dec 2024 21:18:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
3517	2024-12-06T21:18:54.781Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
3518	2024-12-06T21:18:54.783Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
3519	2024-12-06T21:18:54.783Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 376.938122ms iteration = 7 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
3520	2024-12-06T21:18:54.829Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
3521	2024-12-06T21:18:54.829Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
3522	2024-12-06T21:18:54.829Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "4c4759d4-7c19-4961-b05f-35e0304ed9da", "date": "Fri, 06 Dec 2024 21:18:53 GMT"} })
3523	2024-12-06T21:18:54.829Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3524	2024-12-06T21:18:54.829Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3525	2024-12-06T21:18:54.831Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3526	2024-12-06T21:18:54.832Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3527	2024-12-06T21:18:54.834Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3528	2024-12-06T21:18:54.835Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3529	2024-12-06T21:18:54.838Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
3530	2024-12-06T21:18:54.838Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
3531	2024-12-06T21:18:54.838Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "32d432a1-931b-4964-9482-dfcd1d69f4c7", "date": "Fri, 06 Dec 2024 21:18:53 GMT"} })
3532	2024-12-06T21:18:54.840Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 433.924967ms iteration = 9 status = {}
3533	2024-12-06T21:18:54.854Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
3534	2024-12-06T21:18:54.854Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd9389d1-89b9-40ee-adce-3f2d33e72dee", "content-length": "691", "date": "Fri, 06 Dec 2024 21:18:53 GMT"} })
3535	2024-12-06T21:18:54.854Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
3536	2024-12-06T21:18:54.854Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
3537	2024-12-06T21:18:54.854Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
3538	2024-12-06T21:18:54.856Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 450.490299ms iteration = 9 status = {}
3539	2024-12-06T21:18:54.871Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
3540	2024-12-06T21:18:54.990Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
3541	2024-12-06T21:18:54.990Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "47217ddf-b737-48f3-968e-0943821a540e", "date": "Fri, 06 Dec 2024 21:18:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
3542	2024-12-06T21:18:54.990Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
3543	2024-12-06T21:18:54.992Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "87021d6c-65d1-4cb0-8e31-a9ef460502a5", "date": "Fri, 06 Dec 2024 21:18:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
3544	2024-12-06T21:18:54.992Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
3545	2024-12-06T21:18:54.994Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "17b0a86b-b6e6-42e3-9c13-c355ec430993", "date": "Fri, 06 Dec 2024 21:18:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
3546	2024-12-06T21:18:54.994Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
3547	2024-12-06T21:18:54.995Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 588.95346ms iteration = 9 status = {}
3548	2024-12-06T21:18:54.996Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
3549	2024-12-06T21:18:54.996Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 590.299222ms iteration = 7 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
3550	2024-12-06T21:18:55.001Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
3551	2024-12-06T21:18:55.001Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "8cb9d4cd-fff2-4d70-8111-5a12771ec8ec", "content-length": "1", "date": "Fri, 06 Dec 2024 21:18:54 GMT"} })
3552	2024-12-06T21:18:55.002Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3553	2024-12-06T21:18:55.003Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3554	2024-12-06T21:18:55.005Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 598.891303ms iteration = 9 status = null
3555	2024-12-06T21:18:55.034Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3556	2024-12-06T21:18:55.035Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3557	2024-12-06T21:18:55.036Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 3d56b7bf-53ed-4147-b45b-9cef154d7338, rack_id: 2d75e4a3-6363-4fd1-935e-989d31fd7d71, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(02859c9b-92d7-4b19-b425-8f266763be54) } rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
3558	2024-12-06T21:18:55.036Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3559	2024-12-06T21:18:55.037Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3560	2024-12-06T21:18:55.046Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
3561	2024-12-06T21:18:55.047Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "ebe5dabd-b86d-46f9-aceb-31ac0007f73f", "content-length": "134", "date": "Fri, 06 Dec 2024 21:18:54 GMT"} })
3562	2024-12-06T21:18:55.047Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
3563	2024-12-06T21:18:55.047Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 02859c9b-92d7-4b19-b425-8f266763be54
3564	2024-12-06T21:18:55.047Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
3565	2024-12-06T21:18:55.047Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
3566	2024-12-06T21:18:55.047Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3567	2024-12-06T21:18:55.047Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3568	2024-12-06T21:18:55.047Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3569	2024-12-06T21:18:55.047Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
3570	2024-12-06T21:18:55.047Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3571	2024-12-06T21:18:55.047Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
3572	2024-12-06T21:18:55.047Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
3573	2024-12-06T21:18:55.047Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3574	2024-12-06T21:18:55.048Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3575	2024-12-06T21:18:55.049Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
3576	2024-12-06T21:18:55.049Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "5cb54038-50ec-4760-b9a3-85a668779567", "content-length": "2", "date": "Fri, 06 Dec 2024 21:18:54 GMT"} })
3577	2024-12-06T21:18:55.049Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
3578	2024-12-06T21:18:55.049Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "0bd86773-e8b7-4ca2-8182-3c60691b75be", "content-length": "52", "date": "Fri, 06 Dec 2024 21:18:55 GMT"} })
3579	2024-12-06T21:18:55.049Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
3580	2024-12-06T21:18:55.049Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
3581	2024-12-06T21:18:55.231Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "99a9ac43-0f71-4bbd-9e45-3b1fc648375d", "date": "Fri, 06 Dec 2024 21:18:55 GMT"} })
3582	2024-12-06T21:18:55.231Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch = Switch0
3583	2024-12-06T21:18:55.231Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
3584	2024-12-06T21:18:55.232Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "869d0092-fcae-4379-b65c-e511a88c3634", "content-length": "682", "date": "Fri, 06 Dec 2024 21:18:55 GMT"} })
3585	2024-12-06T21:18:55.232Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3586	2024-12-06T21:18:55.232Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3587	2024-12-06T21:18:55.236Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3588	2024-12-06T21:18:55.236Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3589	2024-12-06T21:18:55.239Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3590	2024-12-06T21:18:55.239Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3591	2024-12-06T21:18:55.241Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3592	2024-12-06T21:18:55.241Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3593	2024-12-06T21:18:55.248Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 842.360816ms iteration = 9 status = {}
3594	2024-12-06T21:18:56.541Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 322 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = d6840e81-96a0-45b5-b34d-b272a472eec2 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
3595	2024-12-06T21:19:04.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 26 reason = Timeout
3596	2024-12-06T21:19:04.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 26 reason = Timeout
3597	2024-12-06T21:19:04.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3598	2024-12-06T21:19:04.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3599	2024-12-06T21:19:04.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3600	2024-12-06T21:19:04.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3601	2024-12-06T21:19:04.408Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3602	2024-12-06T21:19:04.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3603	2024-12-06T21:19:04.409Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3604	2024-12-06T21:19:04.410Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.311751ms iteration = 26 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
3605	2024-12-06T21:19:04.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3606	2024-12-06T21:19:04.412Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3607	2024-12-06T21:19:04.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3608	2024-12-06T21:19:04.413Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3609	2024-12-06T21:19:04.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3610	2024-12-06T21:19:04.415Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3611	2024-12-06T21:19:04.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
3612	2024-12-06T21:19:04.418Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3613	2024-12-06T21:19:04.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3614	2024-12-06T21:19:04.428Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
3615	2024-12-06T21:19:04.428Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.489162ms iteration = 26 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
3616	2024-12-06T21:19:06.540Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 288 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = caf1d45c-6c1f-4b3b-af63-c131afb97426 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
3617	2024-12-06T21:19:14.405Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 27 reason = Timeout
3618	2024-12-06T21:19:14.405Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 27 reason = Timeout
3619	2024-12-06T21:19:14.405Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3620	2024-12-06T21:19:14.405Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3621	2024-12-06T21:19:14.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3622	2024-12-06T21:19:14.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3623	2024-12-06T21:19:14.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3624	2024-12-06T21:19:14.408Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3625	2024-12-06T21:19:14.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3626	2024-12-06T21:19:14.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3627	2024-12-06T21:19:14.410Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3628	2024-12-06T21:19:14.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3629	2024-12-06T21:19:14.412Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3630	2024-12-06T21:19:14.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3631	2024-12-06T21:19:14.414Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3632	2024-12-06T21:19:14.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
3633	2024-12-06T21:19:14.416Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3634	2024-12-06T21:19:14.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3635	2024-12-06T21:19:14.428Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
3636	2024-12-06T21:19:14.428Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.018968ms iteration = 27 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
3637	2024-12-06T21:19:14.439Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 33.432289ms iteration = 27 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
3638	2024-12-06T21:19:16.541Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 444 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = e23c8140-2fc4-46a9-bfb3-827a9a1ff51b response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
3639	2024-12-06T21:19:21.535Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = 76512d88-b3e4-466e-972c-400254df2777 roles = RoleSet { roles: {} } uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
3640	2024-12-06T21:19:21.535Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = 76512d88-b3e4-466e-972c-400254df2777 resource = Database result = Ok(()) uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
3641	2024-12-06T21:19:21.537Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2472 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = 76512d88-b3e4-466e-972c-400254df2777 response_code = 200 uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
3642	2024-12-06T21:19:21.537Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = 4fee1bef-b524-45db-8ae2-4824d1510d32 roles = RoleSet { roles: {} } uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
3643	2024-12-06T21:19:21.538Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = 4fee1bef-b524-45db-8ae2-4824d1510d32 resource = Database result = Ok(()) uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
3644	2024-12-06T21:19:21.540Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2880 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = 4fee1bef-b524-45db-8ae2-4824d1510d32 response_code = 200 uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
3645	2024-12-06T21:19:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 10 reason = Timeout
3646	2024-12-06T21:19:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
3647	2024-12-06T21:19:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 10 reason = Timeout
3648	2024-12-06T21:19:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
3649	2024-12-06T21:19:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 10 reason = Timeout
3650	2024-12-06T21:19:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
3651	2024-12-06T21:19:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 10 reason = Timeout
3652	2024-12-06T21:19:24.406Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
3653	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 10 reason = Timeout
3654	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
3655	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3656	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 10 reason = Timeout
3657	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
3658	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 10 reason = Timeout
3659	2024-12-06T21:19:24.407Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
3660	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 11 reason = Timeout
3661	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3662	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 10 reason = Timeout
3663	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3664	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3665	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 10 reason = Timeout
3666	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3667	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 10 reason = Timeout
3668	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 28 reason = Timeout
3669	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 10 reason = Timeout
3670	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
3671	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3672	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 10 reason = Timeout
3673	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 10 reason = Timeout
3674	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
3675	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3676	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 10 reason = Timeout
3677	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
3678	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 28 reason = Timeout
3679	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3680	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3681	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
3682	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3683	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
3684	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3685	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3686	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
3687	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3688	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
3689	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
3690	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3691	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3692	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
3693	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
3694	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
3695	2024-12-06T21:19:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3696	2024-12-06T21:19:24.409Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.587232ms iteration = 10 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
3697	2024-12-06T21:19:24.409Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
3698	2024-12-06T21:19:24.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
3699	2024-12-06T21:19:24.410Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3700	2024-12-06T21:19:24.411Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3701	2024-12-06T21:19:24.411Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
3702	2024-12-06T21:19:24.412Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3703	2024-12-06T21:19:24.412Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1079, tv_nsec: 910588263 } })
3704	2024-12-06T21:19:24.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3705	2024-12-06T21:19:24.412Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
3706	2024-12-06T21:19:24.413Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
3707	2024-12-06T21:19:24.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
3708	2024-12-06T21:19:24.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
3709	2024-12-06T21:19:24.413Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3710	2024-12-06T21:19:24.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3711	2024-12-06T21:19:24.414Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
3712	2024-12-06T21:19:24.415Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
3713	2024-12-06T21:19:24.415Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
3714	2024-12-06T21:19:24.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
3715	2024-12-06T21:19:24.415Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3716	2024-12-06T21:19:24.415Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3717	2024-12-06T21:19:24.416Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1079, tv_nsec: 914653100 } })
3718	2024-12-06T21:19:24.416Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 9.314472ms iteration = 10 status = {"errors":[],"garbage_collect_requested":[]}
3719	2024-12-06T21:19:24.416Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
3720	2024-12-06T21:19:24.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
3721	2024-12-06T21:19:24.417Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
3722	2024-12-06T21:19:24.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3723	2024-12-06T21:19:24.417Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3724	2024-12-06T21:19:24.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
3725	2024-12-06T21:19:24.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3726	2024-12-06T21:19:24.419Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3727	2024-12-06T21:19:24.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3728	2024-12-06T21:19:24.419Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3729	2024-12-06T21:19:24.420Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 13.177416ms iteration = 10 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
3730	2024-12-06T21:19:24.421Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3731	2024-12-06T21:19:24.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3732	2024-12-06T21:19:24.422Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
3733	2024-12-06T21:19:24.422Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3734	2024-12-06T21:19:24.422Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
3735	2024-12-06T21:19:24.422Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
3736	2024-12-06T21:19:24.422Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
3737	2024-12-06T21:19:24.422Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3738	2024-12-06T21:19:24.422Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
3739	2024-12-06T21:19:24.424Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3740	2024-12-06T21:19:24.424Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3741	2024-12-06T21:19:24.424Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3742	2024-12-06T21:19:24.426Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
3743	2024-12-06T21:19:24.426Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3744	2024-12-06T21:19:24.426Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3745	2024-12-06T21:19:24.430Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3746	2024-12-06T21:19:24.435Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
3747	2024-12-06T21:19:24.435Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3748	2024-12-06T21:19:24.435Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3749	2024-12-06T21:19:24.441Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3750	2024-12-06T21:19:24.443Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3751	2024-12-06T21:19:24.443Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
3752	2024-12-06T21:19:24.443Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3753	2024-12-06T21:19:24.443Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3754	2024-12-06T21:19:24.445Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3755	2024-12-06T21:19:24.445Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3756	2024-12-06T21:19:24.446Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
3757	2024-12-06T21:19:24.446Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3758	2024-12-06T21:19:24.446Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3759	2024-12-06T21:19:24.450Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 43.163148ms iteration = 10 status = {"errors":[],"records_set_to_done":[]}
3760	2024-12-06T21:19:24.451Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
3761	2024-12-06T21:19:24.451Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
3762	2024-12-06T21:19:24.453Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
3763	2024-12-06T21:19:24.453Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
3764	2024-12-06T21:19:24.459Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
3765	2024-12-06T21:19:24.459Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
3766	2024-12-06T21:19:24.459Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 52.780156ms iteration = 10 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
3767	2024-12-06T21:19:24.466Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
3768	2024-12-06T21:19:24.466Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3769	2024-12-06T21:19:24.466Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d569970d-bc2d-43f7-b60a-ee3387cf2fb3", "content-length": "26", "date": "Fri, 06 Dec 2024 21:19:24 GMT"} })
3770	2024-12-06T21:19:24.466Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3771	2024-12-06T21:19:24.467Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
3772	2024-12-06T21:19:24.467Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 60.208297ms iteration = 10 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
3773	2024-12-06T21:19:24.468Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3774	2024-12-06T21:19:24.482Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
3775	2024-12-06T21:19:24.482Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3776	2024-12-06T21:19:24.483Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "63c0c652-77ef-47ed-86dd-71e30843487d", "content-length": "26", "date": "Fri, 06 Dec 2024 21:19:24 GMT"} })
3777	2024-12-06T21:19:24.483Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3778	2024-12-06T21:19:24.485Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3779	2024-12-06T21:19:24.495Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3780	2024-12-06T21:19:24.495Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3781	2024-12-06T21:19:24.509Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
3782	2024-12-06T21:19:24.509Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
3783	2024-12-06T21:19:24.510Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "f6d372ae-d322-41c8-9f10-4c388c8f6221", "content-length": "372", "date": "Fri, 06 Dec 2024 21:19:23 GMT"} })
3784	2024-12-06T21:19:24.510Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3785	2024-12-06T21:19:24.510Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3786	2024-12-06T21:19:24.513Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3787	2024-12-06T21:19:24.514Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3788	2024-12-06T21:19:24.514Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1080, tv_nsec: 12897841 } })
3789	2024-12-06T21:19:24.514Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
3790	2024-12-06T21:19:24.526Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
3791	2024-12-06T21:19:24.526Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 119.707134ms iteration = 28 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
3792	2024-12-06T21:19:24.529Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 122.310406ms iteration = 11 status = {"physical_disks_added":0}
3793	2024-12-06T21:19:24.542Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3794	2024-12-06T21:19:24.542Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3795	2024-12-06T21:19:24.544Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3796	2024-12-06T21:19:24.544Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3797	2024-12-06T21:19:24.545Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 137.950293ms iteration = 28 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
3798	2024-12-06T21:19:24.545Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3799	2024-12-06T21:19:24.546Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 139.070151ms iteration = 10 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
3800	2024-12-06T21:19:24.548Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cbc5c1c6-5668-455a-85ef-94f4aebe0f80, lookup_type: ById(cbc5c1c6-5668-455a-85ef-94f4aebe0f80) } result = Ok(())
3801	2024-12-06T21:19:24.548Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3802	2024-12-06T21:19:24.548Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3803	2024-12-06T21:19:24.550Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 143.604705ms iteration = 10 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
3804	2024-12-06T21:19:24.550Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 143.714507ms iteration = 10 status = 0
3805	2024-12-06T21:19:24.552Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3806	2024-12-06T21:19:24.552Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
3807	2024-12-06T21:19:24.552Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "97d871d2-fd82-4a08-a37c-0f726b10be50", "content-length": "1", "date": "Fri, 06 Dec 2024 21:19:24 GMT"} })
3808	2024-12-06T21:19:24.554Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
3809	2024-12-06T21:19:24.554Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3810	2024-12-06T21:19:24.554Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3811	2024-12-06T21:19:24.554Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3812	2024-12-06T21:19:24.555Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3813	2024-12-06T21:19:24.555Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3814	2024-12-06T21:19:24.556Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 149.622394ms iteration = 10 status = null
3815	2024-12-06T21:19:24.557Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
3816	2024-12-06T21:19:24.557Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3817	2024-12-06T21:19:24.557Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3818	2024-12-06T21:19:24.559Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
3819	2024-12-06T21:19:24.559Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
3820	2024-12-06T21:19:24.560Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "1090d26b-a7d4-487e-8f29-e48f2c98d943", "date": "Fri, 06 Dec 2024 21:19:23 GMT"} })
3821	2024-12-06T21:19:24.560Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3822	2024-12-06T21:19:24.560Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3823	2024-12-06T21:19:24.561Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 154.586605ms iteration = 10 status = {}
3824	2024-12-06T21:19:24.561Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3825	2024-12-06T21:19:24.561Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3826	2024-12-06T21:19:24.564Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3827	2024-12-06T21:19:24.565Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3828	2024-12-06T21:19:24.568Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
3829	2024-12-06T21:19:24.568Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
3830	2024-12-06T21:19:24.569Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "88ae7746-6eb9-464f-963c-2f30b140b627", "date": "Fri, 06 Dec 2024 21:19:23 GMT"} })
3831	2024-12-06T21:19:24.570Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 163.731045ms iteration = 10 status = {}
3832	2024-12-06T21:19:24.576Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 zone_address = fd00:1122:3344:101::2
3833	2024-12-06T21:19:24.576Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3834	2024-12-06T21:19:24.576Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3db2ecb-c21e-486b-8b93-421fc5025b16", "content-length": "26", "date": "Fri, 06 Dec 2024 21:19:24 GMT"} })
3835	2024-12-06T21:19:24.576Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3836	2024-12-06T21:19:24.578Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
3837	2024-12-06T21:19:24.689Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
3838	2024-12-06T21:19:24.689Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "eede025a-9559-4ef6-8a27-218b8081537e", "content-length": "691", "date": "Fri, 06 Dec 2024 21:19:24 GMT"} })
3839	2024-12-06T21:19:24.689Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
3840	2024-12-06T21:19:24.689Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
3841	2024-12-06T21:19:24.689Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
3842	2024-12-06T21:19:24.690Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 283.901257ms iteration = 10 status = {}
3843	2024-12-06T21:19:24.715Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3844	2024-12-06T21:19:24.715Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3845	2024-12-06T21:19:24.717Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 3d56b7bf-53ed-4147-b45b-9cef154d7338, rack_id: 2d75e4a3-6363-4fd1-935e-989d31fd7d71, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(02859c9b-92d7-4b19-b425-8f266763be54) } rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
3846	2024-12-06T21:19:24.717Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3847	2024-12-06T21:19:24.717Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3848	2024-12-06T21:19:24.728Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
3849	2024-12-06T21:19:24.728Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "962a85bc-d9ac-44ca-9a7d-a0c9dc773202", "content-length": "134", "date": "Fri, 06 Dec 2024 21:19:24 GMT"} })
3850	2024-12-06T21:19:24.728Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
3851	2024-12-06T21:19:24.728Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 02859c9b-92d7-4b19-b425-8f266763be54
3852	2024-12-06T21:19:24.728Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
3853	2024-12-06T21:19:24.728Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
3854	2024-12-06T21:19:24.728Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3855	2024-12-06T21:19:24.728Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3856	2024-12-06T21:19:24.728Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3857	2024-12-06T21:19:24.728Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
3858	2024-12-06T21:19:24.729Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3859	2024-12-06T21:19:24.729Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
3860	2024-12-06T21:19:24.729Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
3861	2024-12-06T21:19:24.729Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3862	2024-12-06T21:19:24.729Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3863	2024-12-06T21:19:24.731Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
3864	2024-12-06T21:19:24.731Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6103432-0dd6-4127-905c-b2eb466b4728", "content-length": "2", "date": "Fri, 06 Dec 2024 21:19:24 GMT"} })
3865	2024-12-06T21:19:24.731Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
3866	2024-12-06T21:19:24.731Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "1537ae7a-c742-4530-82ac-cd3914ca3c45", "content-length": "52", "date": "Fri, 06 Dec 2024 21:19:24 GMT"} })
3867	2024-12-06T21:19:24.731Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
3868	2024-12-06T21:19:24.731Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
3869	2024-12-06T21:19:24.837Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "003ef0f6-a7b1-4b5f-835d-80b06c4b7c9d", "date": "Fri, 06 Dec 2024 21:19:24 GMT"} })
3870	2024-12-06T21:19:24.837Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch = Switch0
3871	2024-12-06T21:19:24.837Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
3872	2024-12-06T21:19:24.837Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6a3771f-23dc-4a84-b1ad-48425dc357d3", "content-length": "682", "date": "Fri, 06 Dec 2024 21:19:24 GMT"} })
3873	2024-12-06T21:19:24.837Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3874	2024-12-06T21:19:24.838Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3875	2024-12-06T21:19:24.842Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3876	2024-12-06T21:19:24.842Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3877	2024-12-06T21:19:24.845Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3878	2024-12-06T21:19:24.845Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3879	2024-12-06T21:19:24.847Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3880	2024-12-06T21:19:24.847Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3881	2024-12-06T21:19:24.855Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 448.445405ms iteration = 10 status = {}
3882	2024-12-06T21:19:26.540Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 334 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 09a85dbb-3578-49b4-8fc3-29f26eb5df7b response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
3883	2024-12-06T21:19:34.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 29 reason = Timeout
3884	2024-12-06T21:19:34.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 29 reason = Timeout
3885	2024-12-06T21:19:34.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3886	2024-12-06T21:19:34.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3887	2024-12-06T21:19:34.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3888	2024-12-06T21:19:34.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3889	2024-12-06T21:19:34.408Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3890	2024-12-06T21:19:34.408Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3891	2024-12-06T21:19:34.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3892	2024-12-06T21:19:34.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3893	2024-12-06T21:19:34.410Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3894	2024-12-06T21:19:34.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3895	2024-12-06T21:19:34.412Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3896	2024-12-06T21:19:34.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3897	2024-12-06T21:19:34.413Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3898	2024-12-06T21:19:34.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
3899	2024-12-06T21:19:34.415Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3900	2024-12-06T21:19:34.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3901	2024-12-06T21:19:34.427Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
3902	2024-12-06T21:19:34.427Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.388701ms iteration = 29 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
3903	2024-12-06T21:19:34.438Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.738677ms iteration = 29 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
3904	2024-12-06T21:19:36.540Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 305 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = e54be6d7-31dc-4489-bd42-0d5e3895bae2 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
3905	2024-12-06T21:19:44.405Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 30 reason = Timeout
3906	2024-12-06T21:19:44.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 30 reason = Timeout
3907	2024-12-06T21:19:44.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3908	2024-12-06T21:19:44.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3909	2024-12-06T21:19:44.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3910	2024-12-06T21:19:44.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3911	2024-12-06T21:19:44.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3912	2024-12-06T21:19:44.408Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3913	2024-12-06T21:19:44.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3914	2024-12-06T21:19:44.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3915	2024-12-06T21:19:44.410Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3916	2024-12-06T21:19:44.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3917	2024-12-06T21:19:44.412Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3918	2024-12-06T21:19:44.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3919	2024-12-06T21:19:44.413Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3920	2024-12-06T21:19:44.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
3921	2024-12-06T21:19:44.416Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3922	2024-12-06T21:19:44.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3923	2024-12-06T21:19:44.429Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
3924	2024-12-06T21:19:44.429Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.575127ms iteration = 30 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
3925	2024-12-06T21:19:44.436Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 30.821286ms iteration = 30 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
3926	2024-12-06T21:19:46.541Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 270 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = eec0d694-e078-4399-b9cf-dab224f5891b response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
3927	2024-12-06T21:19:51.535Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = 9224abeb-82d0-4c8c-8c29-6a1a95084201 roles = RoleSet { roles: {} } uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
3928	2024-12-06T21:19:51.535Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = 9224abeb-82d0-4c8c-8c29-6a1a95084201 resource = Database result = Ok(()) uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
3929	2024-12-06T21:19:51.538Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3008 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = 9224abeb-82d0-4c8c-8c29-6a1a95084201 response_code = 200 uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
3930	2024-12-06T21:19:51.538Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = 96d3397e-5edf-4489-a03f-8309f7c37af0 roles = RoleSet { roles: {} } uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
3931	2024-12-06T21:19:51.538Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = 96d3397e-5edf-4489-a03f-8309f7c37af0 resource = Database result = Ok(()) uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
3932	2024-12-06T21:19:51.541Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2817 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60694 req_id = 96d3397e-5edf-4489-a03f-8309f7c37af0 response_code = 200 uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
3933	2024-12-06T21:19:54.405Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 8 reason = Timeout
3934	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 6 reason = Timeout
3935	2024-12-06T21:19:54.406Z	INFO	nexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-06 21:09:54.406005180 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
3936	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 7 reason = Timeout
3937	2024-12-06T21:19:54.406Z	WARN	nexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0
3938	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
3939	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 16.98\u{b5}s iteration = 7 status = {"enabled":false,"target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0"}
3940	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 6 reason = Timeout
3941	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 6 reason = Timeout
3942	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
3943	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 6 reason = Timeout
3944	2024-12-06T21:19:54.406Z	INFO	nexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
3945	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 11 reason = Timeout
3946	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3947	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
3948	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 31 reason = Timeout
3949	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 11 reason = Timeout
3950	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3951	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 11 reason = Timeout
3952	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3953	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
3954	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 12 reason = Timeout
3955	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3956	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = external_endpoints iteration = 6 reason = Timeout
3957	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3958	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 11 reason = Timeout
3959	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 8 reason = Timeout
3960	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 11 reason = Timeout
3961	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
3962	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 11 reason = Timeout
3963	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
3964	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 11 reason = Timeout
3965	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
3966	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
3967	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 2 reason = Timeout
3968	2024-12-06T21:19:54.406Z	DEBG	nexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
3969	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3970	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 6 reason = Timeout
3971	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
3972	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 6 reason = Timeout
3973	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 11 reason = Timeout
3974	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
3975	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 11 reason = Timeout
3976	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
3977	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 6 reason = Timeout
3978	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
3979	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3980	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 11 reason = Timeout
3981	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
3982	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 6 reason = Timeout
3983	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
3984	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 11 reason = Timeout
3985	2024-12-06T21:19:54.406Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
3986	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 31 reason = Timeout
3987	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3988	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 11 reason = Timeout
3989	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3990	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 11 reason = Timeout
3991	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 6 reason = Timeout
3992	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3993	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
3994	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3995	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
3996	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 11 reason = Timeout
3997	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3998	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
3999	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4000	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4001	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
4002	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4003	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
4004	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
4005	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
4006	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4007	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
4008	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
4009	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4010	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4011	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
4012	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
4013	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4014	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4015	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
4016	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
4017	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4018	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4019	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
4020	2024-12-06T21:19:54.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
4021	2024-12-06T21:19:54.406Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("709da000-e0da-4b29-8160-3112fd080927.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a03854a6-7a78-402b-93b4-65c6d36bf236.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a03854a6-7a78-402b-93b4-65c6d36bf236.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1109, tv_nsec: 905472995 } })
4022	2024-12-06T21:19:54.407Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4aff1ae3-2d71-40bf-bc46-79a7276da3c2.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4dccdb26-9cf9-40aa-9229-5d35905cf0e1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("79d475dc-44ec-44f4-9e51-ea8a753c39f3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("79d475dc-44ec-44f4-9e51-ea8a753c39f3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1109, tv_nsec: 905480706 } })
4023	2024-12-06T21:19:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4024	2024-12-06T21:19:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4025	2024-12-06T21:19:54.407Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
4026	2024-12-06T21:19:54.407Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.189589ms iteration = 6 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
4027	2024-12-06T21:19:54.407Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
4028	2024-12-06T21:19:54.407Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.505505ms iteration = 6 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
4029	2024-12-06T21:19:54.409Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1109, tv_nsec: 908401343 } })
4030	2024-12-06T21:19:54.410Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.888554ms iteration = 11 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
4031	2024-12-06T21:19:54.410Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4032	2024-12-06T21:19:54.410Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
4033	2024-12-06T21:19:54.410Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4034	2024-12-06T21:19:54.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
4035	2024-12-06T21:19:54.412Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4036	2024-12-06T21:19:54.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4037	2024-12-06T21:19:54.413Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
4038	2024-12-06T21:19:54.413Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4039	2024-12-06T21:19:54.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4040	2024-12-06T21:19:54.414Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 8.153704ms iteration = 11 status = {"errors":[],"garbage_collect_requested":[]}
4041	2024-12-06T21:19:54.415Z	DEBG	nexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 9.507706ms iteration = 6 status = {"errors":[],"found_port_ok":[]}
4042	2024-12-06T21:19:54.416Z	INFO	nexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-06 21:09:54.406005180 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
4043	2024-12-06T21:19:54.416Z	DEBG	nexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 10.705665ms iteration = 6 status = {"expiration":"2024-12-06T21:09:54.406005180Z","pruned":[]}
4044	2024-12-06T21:19:54.418Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4045	2024-12-06T21:19:54.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4046	2024-12-06T21:19:54.418Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4047	2024-12-06T21:19:54.420Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
4048	2024-12-06T21:19:54.420Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
4049	2024-12-06T21:19:54.420Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
4050	2024-12-06T21:19:54.420Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
4051	2024-12-06T21:19:54.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
4052	2024-12-06T21:19:54.421Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
4053	2024-12-06T21:19:54.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
4054	2024-12-06T21:19:54.423Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4055	2024-12-06T21:19:54.424Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4056	2024-12-06T21:19:54.425Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4057	2024-12-06T21:19:54.425Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 19.047542ms iteration = 11 status = {"errors":[],"records_set_to_done":[]}
4058	2024-12-06T21:19:54.425Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4059	2024-12-06T21:19:54.426Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4060	2024-12-06T21:19:54.426Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1109, tv_nsec: 924930325 } })
4061	2024-12-06T21:19:54.426Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
4062	2024-12-06T21:19:54.428Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
4063	2024-12-06T21:19:54.428Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4064	2024-12-06T21:19:54.428Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4065	2024-12-06T21:19:54.435Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4066	2024-12-06T21:19:54.435Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4067	2024-12-06T21:19:54.435Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4068	2024-12-06T21:19:54.436Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4069	2024-12-06T21:19:54.436Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4070	2024-12-06T21:19:54.436Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4071	2024-12-06T21:19:54.438Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4072	2024-12-06T21:19:54.442Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
4073	2024-12-06T21:19:54.442Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4074	2024-12-06T21:19:54.442Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4075	2024-12-06T21:19:54.447Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4076	2024-12-06T21:19:54.450Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
4077	2024-12-06T21:19:54.450Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
4078	2024-12-06T21:19:54.451Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
4079	2024-12-06T21:19:54.451Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4080	2024-12-06T21:19:54.452Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4081	2024-12-06T21:19:54.458Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4082	2024-12-06T21:19:54.461Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
4083	2024-12-06T21:19:54.461Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4084	2024-12-06T21:19:54.461Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4085	2024-12-06T21:19:54.461Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
4086	2024-12-06T21:19:54.461Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4087	2024-12-06T21:19:54.462Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
4088	2024-12-06T21:19:54.463Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
4089	2024-12-06T21:19:54.463Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
4090	2024-12-06T21:19:54.463Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
4091	2024-12-06T21:19:54.463Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4092	2024-12-06T21:19:54.463Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4093	2024-12-06T21:19:54.464Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4094	2024-12-06T21:19:54.467Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
4095	2024-12-06T21:19:54.467Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4096	2024-12-06T21:19:54.467Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4097	2024-12-06T21:19:54.473Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4098	2024-12-06T21:19:54.473Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4099	2024-12-06T21:19:54.476Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1109, tv_nsec: 975061307 } })
4100	2024-12-06T21:19:54.476Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
4101	2024-12-06T21:19:54.477Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4102	2024-12-06T21:19:54.478Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4103	2024-12-06T21:19:54.478Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4104	2024-12-06T21:19:54.479Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4105	2024-12-06T21:19:54.479Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4106	2024-12-06T21:19:54.480Z	DEBG	nexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
4107	2024-12-06T21:19:54.480Z	INFO	nexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
4108	2024-12-06T21:19:54.480Z	DEBG	nexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 73.706799ms iteration = 6 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
4109	2024-12-06T21:19:54.481Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4110	2024-12-06T21:19:54.481Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4111	2024-12-06T21:19:54.482Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4112	2024-12-06T21:19:54.483Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 76.768089ms iteration = 31 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
4113	2024-12-06T21:19:54.484Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4114	2024-12-06T21:19:54.484Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4115	2024-12-06T21:19:54.485Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4116	2024-12-06T21:19:54.488Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4117	2024-12-06T21:19:54.488Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
4118	2024-12-06T21:19:54.488Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 82.259699ms iteration = 11 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
4119	2024-12-06T21:19:54.489Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
4120	2024-12-06T21:19:54.489Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4121	2024-12-06T21:19:54.489Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4122	2024-12-06T21:19:54.490Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4123	2024-12-06T21:19:54.492Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
4124	2024-12-06T21:19:54.492Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
4125	2024-12-06T21:19:54.492Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
4126	2024-12-06T21:19:54.492Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
4127	2024-12-06T21:19:54.492Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
4128	2024-12-06T21:19:54.492Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
4129	2024-12-06T21:19:54.492Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
4130	2024-12-06T21:19:54.494Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4131	2024-12-06T21:19:54.494Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4132	2024-12-06T21:19:54.494Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
4133	2024-12-06T21:19:54.496Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4134	2024-12-06T21:19:54.497Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4135	2024-12-06T21:19:54.499Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
4136	2024-12-06T21:19:54.499Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4137	2024-12-06T21:19:54.499Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4138	2024-12-06T21:19:54.499Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
4139	2024-12-06T21:19:54.499Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4140	2024-12-06T21:19:54.500Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4141	2024-12-06T21:19:54.500Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 94.071983ms iteration = 11 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
4142	2024-12-06T21:19:54.502Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4143	2024-12-06T21:19:54.502Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4144	2024-12-06T21:19:54.503Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4145	2024-12-06T21:19:54.505Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
4146	2024-12-06T21:19:54.505Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4147	2024-12-06T21:19:54.505Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4148	2024-12-06T21:19:54.505Z	DEBG	nexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
4149	2024-12-06T21:19:54.505Z	INFO	nexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
4150	2024-12-06T21:19:54.505Z	DEBG	nexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 99.894948ms iteration = 6 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
4151	2024-12-06T21:19:54.506Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
4152	2024-12-06T21:19:54.506Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
4153	2024-12-06T21:19:54.509Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 103.206013ms iteration = 11 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
4154	2024-12-06T21:19:54.519Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
4155	2024-12-06T21:19:54.520Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
4156	2024-12-06T21:19:54.526Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4157	2024-12-06T21:19:54.526Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4158	2024-12-06T21:19:54.526Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4159	2024-12-06T21:19:54.527Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4160	2024-12-06T21:19:54.527Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
4161	2024-12-06T21:19:54.527Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4162	2024-12-06T21:19:54.528Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4163	2024-12-06T21:19:54.528Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a0e8ee4e-15ff-468c-8439-d8005ba6761f", "content-length": "26", "date": "Fri, 06 Dec 2024 21:19:54 GMT"} })
4164	2024-12-06T21:19:54.528Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4165	2024-12-06T21:19:54.528Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4166	2024-12-06T21:19:54.528Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
4167	2024-12-06T21:19:54.529Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4168	2024-12-06T21:19:54.529Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4169	2024-12-06T21:19:54.530Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
4170	2024-12-06T21:19:54.531Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4171	2024-12-06T21:19:54.531Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4172	2024-12-06T21:19:54.531Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4173	2024-12-06T21:19:54.531Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4174	2024-12-06T21:19:54.532Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4175	2024-12-06T21:19:54.533Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
4176	2024-12-06T21:19:54.533Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4177	2024-12-06T21:19:54.533Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4764281a-9ef8-408c-b797-b62a51e86895", "content-length": "26", "date": "Fri, 06 Dec 2024 21:19:54 GMT"} })
4178	2024-12-06T21:19:54.533Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4179	2024-12-06T21:19:54.535Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4180	2024-12-06T21:19:54.535Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4181	2024-12-06T21:19:54.535Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
4182	2024-12-06T21:19:54.536Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4183	2024-12-06T21:19:54.539Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cbc5c1c6-5668-455a-85ef-94f4aebe0f80, lookup_type: ById(cbc5c1c6-5668-455a-85ef-94f4aebe0f80) } result = Ok(())
4184	2024-12-06T21:19:54.539Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4185	2024-12-06T21:19:54.539Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4186	2024-12-06T21:19:54.541Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
4187	2024-12-06T21:19:54.541Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
4188	2024-12-06T21:19:54.541Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
4189	2024-12-06T21:19:54.541Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4190	2024-12-06T21:19:54.541Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4191	2024-12-06T21:19:54.541Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
4192	2024-12-06T21:19:54.541Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
4193	2024-12-06T21:19:54.541Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 135.37158ms iteration = 11 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
4194	2024-12-06T21:19:54.541Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4195	2024-12-06T21:19:54.542Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4196	2024-12-06T21:19:54.542Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 135.737017ms iteration = 11 status = 0
4197	2024-12-06T21:19:54.543Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
4198	2024-12-06T21:19:54.543Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
4199	2024-12-06T21:19:54.543Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
4200	2024-12-06T21:19:54.543Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4201	2024-12-06T21:19:54.543Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4202	2024-12-06T21:19:54.543Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4203	2024-12-06T21:19:54.543Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4204	2024-12-06T21:19:54.544Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4205	2024-12-06T21:19:54.546Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
4206	2024-12-06T21:19:54.546Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4207	2024-12-06T21:19:54.546Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4208	2024-12-06T21:19:54.547Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4209	2024-12-06T21:19:54.548Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4210	2024-12-06T21:19:54.548Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4211	2024-12-06T21:19:54.549Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
4212	2024-12-06T21:19:54.549Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 143.685837ms iteration = 31 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
4213	2024-12-06T21:19:54.550Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 144.108453ms iteration = 11 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
4214	2024-12-06T21:19:54.551Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
4215	2024-12-06T21:19:54.551Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4216	2024-12-06T21:19:54.551Z	DEBG	nexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-06T21:14:50.566199Z, time_modified: 2024-12-06T21:14:50.566199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, db_silo: Silo { identity: SiloIdentity { id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-06T21:15:06.991365Z, time_modified: 2024-12-06T21:15:06.991365Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, db_silo: Silo { identity: SiloIdentity { id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-06T21:15:06.991365Z, time_modified: 2024-12-06T21:15:06.991365Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1" }] }) }
4217	2024-12-06T21:19:54.551Z	DEBG	nexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 145.024299ms iteration = 6 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"683d02e1-6ec8-48bf-98d0-db6c94785f80","tls_certs":["d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1"]}},"default_endpoint":{"silo_id":"683d02e1-6ec8-48bf-98d0-db6c94785f80","tls_certs":["d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
4218	2024-12-06T21:19:54.551Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4219	2024-12-06T21:19:54.553Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4220	2024-12-06T21:19:54.553Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4221	2024-12-06T21:19:54.553Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
4222	2024-12-06T21:19:54.553Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 93c3f865-bc00-42df-bef7-20e39342f84f dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
4223	2024-12-06T21:19:54.553Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4224	2024-12-06T21:19:54.554Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4225	2024-12-06T21:19:54.554Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4226	2024-12-06T21:19:54.554Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4227	2024-12-06T21:19:54.555Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4228	2024-12-06T21:19:54.557Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
4229	2024-12-06T21:19:54.558Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4230	2024-12-06T21:19:54.558Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4231	2024-12-06T21:19:54.560Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 93c3f865-bc00-42df-bef7-20e39342f84f dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
4232	2024-12-06T21:19:54.560Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
4233	2024-12-06T21:19:54.560Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-06 21:15:06.025861 UTC dns_group = internal generation = 1
4234	2024-12-06T21:19:54.560Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 154.569885ms iteration = 6 status = {"generation":1}
4235	2024-12-06T21:19:54.561Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4236	2024-12-06T21:19:54.561Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4237	2024-12-06T21:19:54.584Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4238	2024-12-06T21:19:54.584Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
4239	2024-12-06T21:19:54.584Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 05c53cbc-f66c-491f-af8d-6f291fe21596 dns_zone_name = oxide.test found_so_far = 0 version = 2
4240	2024-12-06T21:19:54.584Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4241	2024-12-06T21:19:54.584Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4242	2024-12-06T21:19:54.587Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4243	2024-12-06T21:19:54.587Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4244	2024-12-06T21:19:54.587Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4245	2024-12-06T21:19:54.598Z	WARN	nexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
4246	2024-12-06T21:19:54.598Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 192.901644ms iteration = 6 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
4247	2024-12-06T21:19:54.599Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4248	2024-12-06T21:19:54.599Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4249	2024-12-06T21:19:54.603Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 197.471209ms iteration = 12 status = {"physical_disks_added":0}
4250	2024-12-06T21:19:54.617Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 zone_address = fd00:1122:3344:101::2
4251	2024-12-06T21:19:54.617Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4252	2024-12-06T21:19:54.617Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "50a7af39-8166-4ab7-a6bc-036c8ce842e0", "content-length": "26", "date": "Fri, 06 Dec 2024 21:19:54 GMT"} })
4253	2024-12-06T21:19:54.617Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4254	2024-12-06T21:19:54.624Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
4255	2024-12-06T21:19:54.692Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 286.197645ms iteration = 11 status = {}
4256	2024-12-06T21:19:54.717Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
4257	2024-12-06T21:19:54.723Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
4258	2024-12-06T21:19:54.724Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "9d856867-2673-4c35-af41-2f33982d8305", "content-length": "1", "date": "Fri, 06 Dec 2024 21:19:53 GMT"} })
4259	2024-12-06T21:19:54.725Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
4260	2024-12-06T21:19:54.725Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4261	2024-12-06T21:19:54.725Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "e52f9965-6f64-4cca-9bb1-2a43fe581db8", "date": "Fri, 06 Dec 2024 21:19:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4262	2024-12-06T21:19:54.725Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4263	2024-12-06T21:19:54.726Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4264	2024-12-06T21:19:54.727Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "b921fff8-6bd0-405a-beda-1ed98dc9fc7f", "date": "Fri, 06 Dec 2024 21:19:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4265	2024-12-06T21:19:54.727Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4266	2024-12-06T21:19:54.729Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4267	2024-12-06T21:19:54.729Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 323.521447ms iteration = 8 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
4268	2024-12-06T21:19:54.753Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
4269	2024-12-06T21:19:54.753Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
4270	2024-12-06T21:19:54.753Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "b6365758-633c-43bb-9c18-338b3a73dd29", "content-length": "372", "date": "Fri, 06 Dec 2024 21:19:54 GMT"} })
4271	2024-12-06T21:19:54.753Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4272	2024-12-06T21:19:54.753Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4273	2024-12-06T21:19:54.755Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4274	2024-12-06T21:19:54.756Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4275	2024-12-06T21:19:54.756Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4276	2024-12-06T21:19:54.756Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4277	2024-12-06T21:19:54.757Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 351.675838ms iteration = 11 status = null
4278	2024-12-06T21:19:54.759Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
4279	2024-12-06T21:19:54.759Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4280	2024-12-06T21:19:54.759Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
4281	2024-12-06T21:19:54.759Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4282	2024-12-06T21:19:54.760Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4283	2024-12-06T21:19:54.760Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4284	2024-12-06T21:19:54.762Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4285	2024-12-06T21:19:54.766Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
4286	2024-12-06T21:19:54.766Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4287	2024-12-06T21:19:54.766Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4288	2024-12-06T21:19:54.766Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 05c53cbc-f66c-491f-af8d-6f291fe21596 dns_zone_name = oxide.test found_so_far = 1 version = 2
4289	2024-12-06T21:19:54.766Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
4290	2024-12-06T21:19:54.766Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-06 21:15:07.045426 UTC dns_group = external generation = 2
4291	2024-12-06T21:19:54.766Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 360.493033ms iteration = 6 status = {"generation":2}
4292	2024-12-06T21:19:54.769Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4293	2024-12-06T21:19:54.770Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4294	2024-12-06T21:19:54.772Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4295	2024-12-06T21:19:54.772Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4296	2024-12-06T21:19:54.775Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4297	2024-12-06T21:19:54.775Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4298	2024-12-06T21:19:54.776Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4299	2024-12-06T21:19:54.777Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4300	2024-12-06T21:19:54.778Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4301	2024-12-06T21:19:54.778Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4302	2024-12-06T21:19:54.797Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
4303	2024-12-06T21:19:54.798Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "8daa3fe9-9c90-4158-bc39-e0a2a397b83d", "date": "Fri, 06 Dec 2024 21:19:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4304	2024-12-06T21:19:54.798Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4305	2024-12-06T21:19:54.800Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "acb7b753-6962-4dbd-b79f-1c65a8ac7529", "date": "Fri, 06 Dec 2024 21:19:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4306	2024-12-06T21:19:54.800Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4307	2024-12-06T21:19:54.802Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "2a057220-d511-4863-ad40-2701116331ac", "date": "Fri, 06 Dec 2024 21:19:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4308	2024-12-06T21:19:54.802Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4309	2024-12-06T21:19:54.804Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4310	2024-12-06T21:19:54.804Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 398.555766ms iteration = 8 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
4311	2024-12-06T21:19:54.809Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
4312	2024-12-06T21:19:54.809Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
4313	2024-12-06T21:19:54.809Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "051d8e8e-c366-4171-aa18-cdb12cf8d93c", "date": "Fri, 06 Dec 2024 21:19:54 GMT"} })
4314	2024-12-06T21:19:54.809Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4315	2024-12-06T21:19:54.810Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4316	2024-12-06T21:19:54.836Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
4317	2024-12-06T21:19:54.836Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "ced292df-8772-42cb-a994-b44ddbd59314", "content-length": "691", "date": "Fri, 06 Dec 2024 21:19:54 GMT"} })
4318	2024-12-06T21:19:54.836Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
4319	2024-12-06T21:19:54.836Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
4320	2024-12-06T21:19:54.836Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
4321	2024-12-06T21:19:54.837Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 431.737121ms iteration = 11 status = {}
4322	2024-12-06T21:19:54.838Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4323	2024-12-06T21:19:54.838Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4324	2024-12-06T21:19:54.839Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4325	2024-12-06T21:19:54.841Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
4326	2024-12-06T21:19:54.841Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4327	2024-12-06T21:19:54.841Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4328	2024-12-06T21:19:54.842Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4329	2024-12-06T21:19:54.842Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4330	2024-12-06T21:19:54.843Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4331	2024-12-06T21:19:54.843Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4332	2024-12-06T21:19:54.845Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4333	2024-12-06T21:19:54.845Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4334	2024-12-06T21:19:54.845Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
4335	2024-12-06T21:19:54.845Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
4336	2024-12-06T21:19:54.845Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "fa6f937d-0d0f-401d-87a3-17626c4e1c03", "date": "Fri, 06 Dec 2024 21:19:54 GMT"} })
4337	2024-12-06T21:19:54.846Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4338	2024-12-06T21:19:54.847Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 441.045414ms iteration = 11 status = {}
4339	2024-12-06T21:19:54.849Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
4340	2024-12-06T21:19:54.849Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4341	2024-12-06T21:19:54.849Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4342	2024-12-06T21:19:54.858Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4343	2024-12-06T21:19:54.858Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4344	2024-12-06T21:19:54.860Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4345	2024-12-06T21:19:54.861Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4346	2024-12-06T21:19:54.862Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4347	2024-12-06T21:19:54.862Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4348	2024-12-06T21:19:54.864Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4349	2024-12-06T21:19:54.864Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4350	2024-12-06T21:19:54.866Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4351	2024-12-06T21:19:54.866Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4352	2024-12-06T21:19:54.866Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4353	2024-12-06T21:19:54.866Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4354	2024-12-06T21:19:54.867Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4355	2024-12-06T21:19:54.867Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4356	2024-12-06T21:19:54.868Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 3d56b7bf-53ed-4147-b45b-9cef154d7338, rack_id: 2d75e4a3-6363-4fd1-935e-989d31fd7d71, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(02859c9b-92d7-4b19-b425-8f266763be54) } rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
4357	2024-12-06T21:19:54.868Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4358	2024-12-06T21:19:54.868Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4359	2024-12-06T21:19:54.868Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4360	2024-12-06T21:19:54.869Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4361	2024-12-06T21:19:54.870Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4362	2024-12-06T21:19:54.875Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
4363	2024-12-06T21:19:54.875Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4364	2024-12-06T21:19:54.875Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4365	2024-12-06T21:19:54.877Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4366	2024-12-06T21:19:54.877Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4367	2024-12-06T21:19:54.878Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
4368	2024-12-06T21:19:54.878Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4369	2024-12-06T21:19:54.878Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "d5e1a1ab-8f83-4ea5-bdec-07f469fe2025", "content-length": "134", "date": "Fri, 06 Dec 2024 21:19:53 GMT"} })
4370	2024-12-06T21:19:54.878Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
4371	2024-12-06T21:19:54.878Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 02859c9b-92d7-4b19-b425-8f266763be54
4372	2024-12-06T21:19:54.878Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4373	2024-12-06T21:19:54.879Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
4374	2024-12-06T21:19:54.879Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
4375	2024-12-06T21:19:54.879Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4376	2024-12-06T21:19:54.879Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4377	2024-12-06T21:19:54.879Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4378	2024-12-06T21:19:54.879Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
4379	2024-12-06T21:19:54.879Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4380	2024-12-06T21:19:54.879Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
4381	2024-12-06T21:19:54.879Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
4382	2024-12-06T21:19:54.879Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4383	2024-12-06T21:19:54.879Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4384	2024-12-06T21:19:54.879Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4385	2024-12-06T21:19:54.880Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
4386	2024-12-06T21:19:54.881Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "2092e6a3-0d54-447a-a3c0-ac7c490d220c", "content-length": "2", "date": "Fri, 06 Dec 2024 21:19:53 GMT"} })
4387	2024-12-06T21:19:54.881Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
4388	2024-12-06T21:19:54.881Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "4673ff7b-7c74-4a54-9be1-19d7e5f41074", "content-length": "52", "date": "Fri, 06 Dec 2024 21:19:53 GMT"} })
4389	2024-12-06T21:19:54.881Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
4390	2024-12-06T21:19:54.881Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
4391	2024-12-06T21:19:54.884Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
4392	2024-12-06T21:19:54.884Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4393	2024-12-06T21:19:54.884Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4394	2024-12-06T21:19:54.893Z	DEBG	nexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 470a6d71-26ee-4d59-b30a-7a9e78dfa903, kind: Service { id: 709da000-e0da-4b29-8160-3112fd080927 }, name: Name("external-dns-709da000-e0da-4b29-8160-3112fd080927"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 160, 53])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8dee8d46-edc9-4f8a-9453-2146d512c47b, kind: Service { id: a03854a6-7a78-402b-93b4-65c6d36bf236 }, name: Name("external-dns-a03854a6-7a78-402b-93b4-65c6d36bf236"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 200])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 112d04a8-d7b2-41ba-a8a9-b1b05b82789a, kind: Service { id: e8725c8c-13ab-4916-bf16-bb7f832014d5 }, name: Name("nexus-e8725c8c-13ab-4916-bf16-bb7f832014d5"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 245, 172])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 3b455a5c-72a6-4c48-bc31-0f3337db4efa, kind: Service { id: a211d4f7-7350-4466-97ca-36b58597b187 }, name: Name("nexus-a211d4f7-7350-4466-97ca-36b58597b187"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 181, 41])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 73f8344b-aadb-4eda-971a-b52bf9a56847, kind: Service { id: 0214b950-30f3-4a60-9344-b4f7d110e307 }, name: Name("nexus-0214b950-30f3-4a60-9344-b4f7d110e307"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 172, 92])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
4395	2024-12-06T21:19:54.893Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4396	2024-12-06T21:19:54.893Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4397	2024-12-06T21:19:54.894Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4398	2024-12-06T21:19:54.896Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
4399	2024-12-06T21:19:54.896Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4400	2024-12-06T21:19:54.896Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4401	2024-12-06T21:19:54.898Z	DEBG	nexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
4402	2024-12-06T21:19:54.898Z	DEBG	nexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 470a6d71-26ee-4d59-b30a-7a9e78dfa903, kind: Service { id: 709da000-e0da-4b29-8160-3112fd080927 }, name: Name("external-dns-709da000-e0da-4b29-8160-3112fd080927"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 160, 53])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8dee8d46-edc9-4f8a-9453-2146d512c47b, kind: Service { id: a03854a6-7a78-402b-93b4-65c6d36bf236 }, name: Name("external-dns-a03854a6-7a78-402b-93b4-65c6d36bf236"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 200])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 112d04a8-d7b2-41ba-a8a9-b1b05b82789a, kind: Service { id: e8725c8c-13ab-4916-bf16-bb7f832014d5 }, name: Name("nexus-e8725c8c-13ab-4916-bf16-bb7f832014d5"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 245, 172])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 3b455a5c-72a6-4c48-bc31-0f3337db4efa, kind: Service { id: a211d4f7-7350-4466-97ca-36b58597b187 }, name: Name("nexus-a211d4f7-7350-4466-97ca-36b58597b187"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 181, 41])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 73f8344b-aadb-4eda-971a-b52bf9a56847, kind: Service { id: 0214b950-30f3-4a60-9344-b4f7d110e307 }, name: Name("nexus-0214b950-30f3-4a60-9344-b4f7d110e307"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 172, 92])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
4403	2024-12-06T21:19:54.898Z	DEBG	nexus (service-firewall-rule-progator): resolved 2 rules for sleds background_task = service_firewall_rule_propagation
4404	2024-12-06T21:19:54.956Z	DEBG	nexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: cbc5c1c6-5668-455a-85ef-94f4aebe0f80, time_created: 2024-12-06T21:15:03.870849Z, time_modified: 2024-12-06T21:15:03.870849Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 2d75e4a3-6363-4fd1-935e-989d31fd7d71, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(51474595840)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)) }]
4405	2024-12-06T21:19:54.956Z	DEBG	nexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
4406	2024-12-06T21:19:54.956Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4407	2024-12-06T21:19:54.956Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4408	2024-12-06T21:19:54.958Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4409	2024-12-06T21:19:54.958Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4410	2024-12-06T21:19:54.960Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4411	2024-12-06T21:19:54.963Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: cbc5c1c6-5668-455a-85ef-94f4aebe0f80, lookup_type: ById(cbc5c1c6-5668-455a-85ef-94f4aebe0f80) } result = Ok(())
4412	2024-12-06T21:19:54.982Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "0f81a2dc-ca26-4dca-ba50-731e0d3dc16c", "date": "Fri, 06 Dec 2024 21:19:54 GMT"} })
4413	2024-12-06T21:19:54.982Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch = Switch0
4414	2024-12-06T21:19:54.982Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
4415	2024-12-06T21:19:54.982Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "c383c72b-4cca-4b1d-973c-e9342bdcd37b", "content-length": "682", "date": "Fri, 06 Dec 2024 21:19:54 GMT"} })
4416	2024-12-06T21:19:54.982Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4417	2024-12-06T21:19:54.983Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4418	2024-12-06T21:19:54.986Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4419	2024-12-06T21:19:54.986Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4420	2024-12-06T21:19:54.990Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4421	2024-12-06T21:19:54.990Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4422	2024-12-06T21:19:54.992Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4423	2024-12-06T21:19:54.993Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4424	2024-12-06T21:19:54.993Z	DEBG	nexus (service-firewall-rule-progator): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
4425	2024-12-06T21:19:54.993Z	DEBG	nexus (service-firewall-rule-progator): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "6ec00280-00e8-4fe6-8be8-428ae373c430", "date": "Fri, 06 Dec 2024 21:19:54 GMT"} })
4426	2024-12-06T21:19:54.997Z	INFO	nexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:496
4427	2024-12-06T21:19:54.997Z	DEBG	nexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 590.975813ms
4428	2024-12-06T21:19:54.997Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 590.986253ms iteration = 2 status = {}
4429	2024-12-06T21:19:55.000Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 593.807459ms iteration = 11 status = {}
4430	2024-12-06T21:19:56.540Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 292 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 57fe460b-3f9c-422a-979f-6793c09d79b7 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
4431	2024-12-06T21:20:04.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 32 reason = Timeout
4432	2024-12-06T21:20:04.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 32 reason = Timeout
4433	2024-12-06T21:20:04.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4434	2024-12-06T21:20:04.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4435	2024-12-06T21:20:04.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4436	2024-12-06T21:20:04.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4437	2024-12-06T21:20:04.408Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4438	2024-12-06T21:20:04.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4439	2024-12-06T21:20:04.409Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4440	2024-12-06T21:20:04.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4441	2024-12-06T21:20:04.412Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4442	2024-12-06T21:20:04.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4443	2024-12-06T21:20:04.414Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4444	2024-12-06T21:20:04.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4445	2024-12-06T21:20:04.416Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4446	2024-12-06T21:20:04.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
4447	2024-12-06T21:20:04.418Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4448	2024-12-06T21:20:04.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4449	2024-12-06T21:20:04.427Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
4450	2024-12-06T21:20:04.427Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.590247ms iteration = 32 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
4451	2024-12-06T21:20:04.438Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.870213ms iteration = 32 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
4452	2024-12-06T21:20:06.541Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 281 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = d7067a4f-49df-4ea7-a5e1-44b7dc9991ba response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
4453	2024-12-06T21:20:14.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 33 reason = Timeout
4454	2024-12-06T21:20:14.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 33 reason = Timeout
4455	2024-12-06T21:20:14.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4456	2024-12-06T21:20:14.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4457	2024-12-06T21:20:14.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4458	2024-12-06T21:20:14.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4459	2024-12-06T21:20:14.408Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4460	2024-12-06T21:20:14.409Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4461	2024-12-06T21:20:14.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4462	2024-12-06T21:20:14.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4463	2024-12-06T21:20:14.411Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4464	2024-12-06T21:20:14.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4465	2024-12-06T21:20:14.413Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4466	2024-12-06T21:20:14.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4467	2024-12-06T21:20:14.415Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4468	2024-12-06T21:20:14.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
4469	2024-12-06T21:20:14.417Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4470	2024-12-06T21:20:14.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4471	2024-12-06T21:20:14.427Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
4472	2024-12-06T21:20:14.427Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.977204ms iteration = 33 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
4473	2024-12-06T21:20:14.438Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.30337ms iteration = 33 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
4474	2024-12-06T21:20:16.541Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 476 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 867044c3-f87c-4bf1-b492-ebdb14d9f938 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
4475	2024-12-06T21:20:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 12 reason = Timeout
4476	2024-12-06T21:20:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
4477	2024-12-06T21:20:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 12 reason = Timeout
4478	2024-12-06T21:20:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 12 reason = Timeout
4479	2024-12-06T21:20:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
4480	2024-12-06T21:20:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
4481	2024-12-06T21:20:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 12 reason = Timeout
4482	2024-12-06T21:20:24.406Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
4483	2024-12-06T21:20:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
4484	2024-12-06T21:20:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 12 reason = Timeout
4485	2024-12-06T21:20:24.406Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
4486	2024-12-06T21:20:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 12 reason = Timeout
4487	2024-12-06T21:20:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
4488	2024-12-06T21:20:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 12 reason = Timeout
4489	2024-12-06T21:20:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4490	2024-12-06T21:20:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
4491	2024-12-06T21:20:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 12 reason = Timeout
4492	2024-12-06T21:20:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4493	2024-12-06T21:20:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 12 reason = Timeout
4494	2024-12-06T21:20:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
4495	2024-12-06T21:20:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 12 reason = Timeout
4496	2024-12-06T21:20:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4497	2024-12-06T21:20:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 12 reason = Timeout
4498	2024-12-06T21:20:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4499	2024-12-06T21:20:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 34 reason = Timeout
4500	2024-12-06T21:20:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4501	2024-12-06T21:20:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 12 reason = Timeout
4502	2024-12-06T21:20:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4503	2024-12-06T21:20:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 12 reason = Timeout
4504	2024-12-06T21:20:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
4505	2024-12-06T21:20:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 34 reason = Timeout
4506	2024-12-06T21:20:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 12 reason = Timeout
4507	2024-12-06T21:20:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4508	2024-12-06T21:20:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 13 reason = Timeout
4509	2024-12-06T21:20:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4510	2024-12-06T21:20:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
4511	2024-12-06T21:20:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
4512	2024-12-06T21:20:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
4513	2024-12-06T21:20:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
4514	2024-12-06T21:20:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
4515	2024-12-06T21:20:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4516	2024-12-06T21:20:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
4517	2024-12-06T21:20:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4518	2024-12-06T21:20:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4519	2024-12-06T21:20:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
4520	2024-12-06T21:20:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4521	2024-12-06T21:20:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4522	2024-12-06T21:20:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
4523	2024-12-06T21:20:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4524	2024-12-06T21:20:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4525	2024-12-06T21:20:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4526	2024-12-06T21:20:24.409Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1139, tv_nsec: 908167812 } })
4527	2024-12-06T21:20:24.409Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.01064ms iteration = 12 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
4528	2024-12-06T21:20:24.410Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
4529	2024-12-06T21:20:24.410Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 3.68427ms iteration = 12 status = {"errors":[],"garbage_collect_requested":[]}
4530	2024-12-06T21:20:24.411Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4531	2024-12-06T21:20:24.412Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1139, tv_nsec: 911405155 } })
4532	2024-12-06T21:20:24.413Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4533	2024-12-06T21:20:24.413Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
4534	2024-12-06T21:20:24.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
4535	2024-12-06T21:20:24.413Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4536	2024-12-06T21:20:24.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4537	2024-12-06T21:20:24.415Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4538	2024-12-06T21:20:24.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
4539	2024-12-06T21:20:24.415Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4540	2024-12-06T21:20:24.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4541	2024-12-06T21:20:24.416Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4542	2024-12-06T21:20:24.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
4543	2024-12-06T21:20:24.417Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
4544	2024-12-06T21:20:24.418Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4545	2024-12-06T21:20:24.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
4546	2024-12-06T21:20:24.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4547	2024-12-06T21:20:24.418Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4548	2024-12-06T21:20:24.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4549	2024-12-06T21:20:24.419Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4550	2024-12-06T21:20:24.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
4551	2024-12-06T21:20:24.419Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4552	2024-12-06T21:20:24.420Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4553	2024-12-06T21:20:24.421Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
4554	2024-12-06T21:20:24.421Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4555	2024-12-06T21:20:24.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4556	2024-12-06T21:20:24.421Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1139, tv_nsec: 920255540 } })
4557	2024-12-06T21:20:24.421Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4558	2024-12-06T21:20:24.422Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
4559	2024-12-06T21:20:24.422Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4560	2024-12-06T21:20:24.423Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4561	2024-12-06T21:20:24.424Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4562	2024-12-06T21:20:24.424Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4563	2024-12-06T21:20:24.424Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4564	2024-12-06T21:20:24.424Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4565	2024-12-06T21:20:24.426Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
4566	2024-12-06T21:20:24.426Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4567	2024-12-06T21:20:24.426Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4568	2024-12-06T21:20:24.430Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4569	2024-12-06T21:20:24.431Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4570	2024-12-06T21:20:24.431Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
4571	2024-12-06T21:20:24.431Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4572	2024-12-06T21:20:24.432Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4573	2024-12-06T21:20:24.434Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
4574	2024-12-06T21:20:24.434Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4575	2024-12-06T21:20:24.434Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4576	2024-12-06T21:20:24.441Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
4577	2024-12-06T21:20:24.441Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
4578	2024-12-06T21:20:24.444Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 37.436094ms iteration = 12 status = {"errors":[],"records_set_to_done":[]}
4579	2024-12-06T21:20:24.445Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
4580	2024-12-06T21:20:24.446Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
4581	2024-12-06T21:20:24.447Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
4582	2024-12-06T21:20:24.447Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
4583	2024-12-06T21:20:24.447Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
4584	2024-12-06T21:20:24.447Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
4585	2024-12-06T21:20:24.447Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
4586	2024-12-06T21:20:24.447Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 40.8634ms iteration = 34 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
4587	2024-12-06T21:20:24.450Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
4588	2024-12-06T21:20:24.450Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
4589	2024-12-06T21:20:24.452Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 45.490486ms iteration = 12 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
4590	2024-12-06T21:20:24.453Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4591	2024-12-06T21:20:24.454Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4592	2024-12-06T21:20:24.454Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4593	2024-12-06T21:20:24.454Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4594	2024-12-06T21:20:24.457Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 50.830974ms iteration = 34 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
4595	2024-12-06T21:20:24.459Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
4596	2024-12-06T21:20:24.459Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 52.673054ms iteration = 12 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
4597	2024-12-06T21:20:24.460Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4598	2024-12-06T21:20:24.460Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4599	2024-12-06T21:20:24.462Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4600	2024-12-06T21:20:24.466Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cbc5c1c6-5668-455a-85ef-94f4aebe0f80, lookup_type: ById(cbc5c1c6-5668-455a-85ef-94f4aebe0f80) } result = Ok(())
4601	2024-12-06T21:20:24.466Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4602	2024-12-06T21:20:24.466Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4603	2024-12-06T21:20:24.468Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 61.464688ms iteration = 12 status = 0
4604	2024-12-06T21:20:24.479Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
4605	2024-12-06T21:20:24.480Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
4606	2024-12-06T21:20:24.481Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 75.00753ms iteration = 12 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
4607	2024-12-06T21:20:24.488Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
4608	2024-12-06T21:20:24.488Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4609	2024-12-06T21:20:24.488Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d13b5cd9-dfed-4690-9335-5f45c1cba258", "content-length": "26", "date": "Fri, 06 Dec 2024 21:20:24 GMT"} })
4610	2024-12-06T21:20:24.488Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4611	2024-12-06T21:20:24.489Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
4612	2024-12-06T21:20:24.489Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
4613	2024-12-06T21:20:24.489Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
4614	2024-12-06T21:20:24.489Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
4615	2024-12-06T21:20:24.490Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
4616	2024-12-06T21:20:24.493Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 86.918745ms iteration = 12 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
4617	2024-12-06T21:20:24.495Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 zone_address = fd00:1122:3344:101::2
4618	2024-12-06T21:20:24.495Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4619	2024-12-06T21:20:24.495Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "dad01292-8801-40b5-b8db-b034dc1e750f", "content-length": "26", "date": "Fri, 06 Dec 2024 21:20:24 GMT"} })
4620	2024-12-06T21:20:24.495Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4621	2024-12-06T21:20:24.496Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
4622	2024-12-06T21:20:24.502Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
4623	2024-12-06T21:20:24.502Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4624	2024-12-06T21:20:24.502Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e30e75c3-492a-4bf4-a3cf-0b75071ac5d4", "content-length": "26", "date": "Fri, 06 Dec 2024 21:20:24 GMT"} })
4625	2024-12-06T21:20:24.502Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4626	2024-12-06T21:20:24.504Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
4627	2024-12-06T21:20:24.526Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
4628	2024-12-06T21:20:24.526Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
4629	2024-12-06T21:20:24.526Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 119.275276ms iteration = 12 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
4630	2024-12-06T21:20:24.537Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 130.144634ms iteration = 13 status = {"physical_disks_added":0}
4631	2024-12-06T21:20:24.541Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4632	2024-12-06T21:20:24.542Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4633	2024-12-06T21:20:24.549Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 142.940974ms iteration = 12 status = {}
4634	2024-12-06T21:20:24.550Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4635	2024-12-06T21:20:24.550Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4636	2024-12-06T21:20:24.552Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4637	2024-12-06T21:20:24.554Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
4638	2024-12-06T21:20:24.554Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4639	2024-12-06T21:20:24.554Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4640	2024-12-06T21:20:24.556Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4641	2024-12-06T21:20:24.557Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
4642	2024-12-06T21:20:24.557Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4643	2024-12-06T21:20:24.557Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4644	2024-12-06T21:20:24.565Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
4645	2024-12-06T21:20:24.565Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a402afd-1a66-4b3e-a829-185af35bf20f", "content-length": "1", "date": "Fri, 06 Dec 2024 21:20:23 GMT"} })
4646	2024-12-06T21:20:24.566Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4647	2024-12-06T21:20:24.566Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4648	2024-12-06T21:20:24.567Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
4649	2024-12-06T21:20:24.567Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
4650	2024-12-06T21:20:24.568Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "e588130f-3cf3-4558-a636-41edecd09088", "content-length": "372", "date": "Fri, 06 Dec 2024 21:20:23 GMT"} })
4651	2024-12-06T21:20:24.568Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4652	2024-12-06T21:20:24.568Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4653	2024-12-06T21:20:24.569Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 162.814311ms iteration = 12 status = null
4654	2024-12-06T21:20:24.581Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
4655	2024-12-06T21:20:24.581Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
4656	2024-12-06T21:20:24.581Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "207693a1-80c2-4850-bf66-12061161a025", "date": "Fri, 06 Dec 2024 21:20:23 GMT"} })
4657	2024-12-06T21:20:24.581Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4658	2024-12-06T21:20:24.581Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4659	2024-12-06T21:20:24.583Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4660	2024-12-06T21:20:24.583Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4661	2024-12-06T21:20:24.585Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4662	2024-12-06T21:20:24.586Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4663	2024-12-06T21:20:24.588Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
4664	2024-12-06T21:20:24.588Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
4665	2024-12-06T21:20:24.589Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "130dc7ea-72cb-4a43-9536-efe3a15ca2cb", "date": "Fri, 06 Dec 2024 21:20:23 GMT"} })
4666	2024-12-06T21:20:24.590Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 183.874136ms iteration = 12 status = {}
4667	2024-12-06T21:20:24.628Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4668	2024-12-06T21:20:24.628Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4669	2024-12-06T21:20:24.630Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 3d56b7bf-53ed-4147-b45b-9cef154d7338, rack_id: 2d75e4a3-6363-4fd1-935e-989d31fd7d71, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(02859c9b-92d7-4b19-b425-8f266763be54) } rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
4670	2024-12-06T21:20:24.630Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4671	2024-12-06T21:20:24.631Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4672	2024-12-06T21:20:24.652Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
4673	2024-12-06T21:20:24.652Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "719603dc-b20a-407f-81a0-d691cf9857a7", "content-length": "134", "date": "Fri, 06 Dec 2024 21:20:23 GMT"} })
4674	2024-12-06T21:20:24.653Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
4675	2024-12-06T21:20:24.653Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 02859c9b-92d7-4b19-b425-8f266763be54
4676	2024-12-06T21:20:24.653Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
4677	2024-12-06T21:20:24.653Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
4678	2024-12-06T21:20:24.653Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4679	2024-12-06T21:20:24.653Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4680	2024-12-06T21:20:24.653Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4681	2024-12-06T21:20:24.653Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
4682	2024-12-06T21:20:24.653Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4683	2024-12-06T21:20:24.653Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
4684	2024-12-06T21:20:24.653Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
4685	2024-12-06T21:20:24.653Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4686	2024-12-06T21:20:24.654Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4687	2024-12-06T21:20:24.655Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
4688	2024-12-06T21:20:24.655Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "4ee28a1a-348f-48df-8821-0036f9b3360d", "content-length": "2", "date": "Fri, 06 Dec 2024 21:20:23 GMT"} })
4689	2024-12-06T21:20:24.655Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
4690	2024-12-06T21:20:24.655Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "74cb5b74-2721-4bab-a619-2b3787b47c9a", "content-length": "52", "date": "Fri, 06 Dec 2024 21:20:23 GMT"} })
4691	2024-12-06T21:20:24.655Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
4692	2024-12-06T21:20:24.655Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
4693	2024-12-06T21:20:24.664Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
4694	2024-12-06T21:20:24.766Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "baccf568-1fed-49e1-baca-84eb5fddf36b", "date": "Fri, 06 Dec 2024 21:20:24 GMT"} })
4695	2024-12-06T21:20:24.766Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch = Switch0
4696	2024-12-06T21:20:24.766Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
4697	2024-12-06T21:20:24.766Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "79738857-f59e-4528-aa57-53cc05722d04", "content-length": "691", "date": "Fri, 06 Dec 2024 21:20:24 GMT"} })
4698	2024-12-06T21:20:24.766Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
4699	2024-12-06T21:20:24.766Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
4700	2024-12-06T21:20:24.766Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
4701	2024-12-06T21:20:24.766Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "abef7ab3-93a7-45a7-8dce-be7b72103e3b", "content-length": "682", "date": "Fri, 06 Dec 2024 21:20:24 GMT"} })
4702	2024-12-06T21:20:24.766Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4703	2024-12-06T21:20:24.767Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4704	2024-12-06T21:20:24.768Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 361.235295ms iteration = 12 status = {}
4705	2024-12-06T21:20:24.770Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4706	2024-12-06T21:20:24.771Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4707	2024-12-06T21:20:24.773Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4708	2024-12-06T21:20:24.773Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4709	2024-12-06T21:20:24.775Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4710	2024-12-06T21:20:24.775Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4711	2024-12-06T21:20:24.782Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 375.200014ms iteration = 12 status = {}
4712	2024-12-06T21:20:26.540Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 270 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = c59f2ac2-d28b-48ae-9cc0-0447c2ac3e6e response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
4713	2024-12-06T21:20:34.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 35 reason = Timeout
4714	2024-12-06T21:20:34.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 35 reason = Timeout
4715	2024-12-06T21:20:34.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4716	2024-12-06T21:20:34.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4717	2024-12-06T21:20:34.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4718	2024-12-06T21:20:34.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4719	2024-12-06T21:20:34.408Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4720	2024-12-06T21:20:34.408Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4721	2024-12-06T21:20:34.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4722	2024-12-06T21:20:34.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4723	2024-12-06T21:20:34.410Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.481734ms iteration = 35 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
4724	2024-12-06T21:20:34.410Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4725	2024-12-06T21:20:34.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4726	2024-12-06T21:20:34.411Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4727	2024-12-06T21:20:34.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4728	2024-12-06T21:20:34.413Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4729	2024-12-06T21:20:34.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
4730	2024-12-06T21:20:34.416Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4731	2024-12-06T21:20:34.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4732	2024-12-06T21:20:34.425Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
4733	2024-12-06T21:20:34.425Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.172115ms iteration = 35 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
4734	2024-12-06T21:20:36.541Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 253 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 3ec6d581-b98e-4f98-b38e-8e21ab1f07aa response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
4735	2024-12-06T21:20:44.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 36 reason = Timeout
4736	2024-12-06T21:20:44.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 36 reason = Timeout
4737	2024-12-06T21:20:44.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4738	2024-12-06T21:20:44.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4739	2024-12-06T21:20:44.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4740	2024-12-06T21:20:44.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4741	2024-12-06T21:20:44.409Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4742	2024-12-06T21:20:44.409Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4743	2024-12-06T21:20:44.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4744	2024-12-06T21:20:44.410Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.68842ms iteration = 36 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
4745	2024-12-06T21:20:44.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4746	2024-12-06T21:20:44.411Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4747	2024-12-06T21:20:44.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4748	2024-12-06T21:20:44.413Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4749	2024-12-06T21:20:44.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4750	2024-12-06T21:20:44.415Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4751	2024-12-06T21:20:44.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
4752	2024-12-06T21:20:44.417Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4753	2024-12-06T21:20:44.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4754	2024-12-06T21:20:44.427Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
4755	2024-12-06T21:20:44.427Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.655029ms iteration = 36 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
4756	2024-12-06T21:20:46.541Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 286 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 345b7f02-4171-460d-b648-95c4fa9d661e response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
4757	2024-12-06T21:20:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 9 reason = Timeout
4758	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 9 reason = Timeout
4759	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 13 reason = Timeout
4760	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 7 reason = Timeout
4761	2024-12-06T21:20:54.407Z	INFO	nexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
4762	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 13 reason = Timeout
4763	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4764	2024-12-06T21:20:54.407Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
4765	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
4766	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 13 reason = Timeout
4767	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4768	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 7 reason = Timeout
4769	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4770	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
4771	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 4 reason = Timeout
4772	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 13 reason = Timeout
4773	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
4774	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 13 reason = Timeout
4775	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
4776	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 7 reason = Timeout
4777	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
4778	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
4779	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4780	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 7 reason = Timeout
4781	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
4782	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 37 reason = Timeout
4783	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 13 reason = Timeout
4784	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4785	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = external_endpoints iteration = 7 reason = Timeout
4786	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 13 reason = Timeout
4787	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4788	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
4789	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 8 reason = Timeout
4790	2024-12-06T21:20:54.407Z	WARN	nexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0
4791	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 7 reason = Timeout
4792	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 11.34\u{b5}s iteration = 8 status = {"enabled":false,"target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0"}
4793	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 14 reason = Timeout
4794	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 13 reason = Timeout
4795	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4796	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4797	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 37 reason = Timeout
4798	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4799	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 13 reason = Timeout
4800	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4801	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 13 reason = Timeout
4802	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
4803	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 13 reason = Timeout
4804	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
4805	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
4806	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 7 reason = Timeout
4807	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 13 reason = Timeout
4808	2024-12-06T21:20:54.407Z	INFO	nexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-06 21:10:54.407304728 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
4809	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
4810	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
4811	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 7 reason = Timeout
4812	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
4813	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4814	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 7 reason = Timeout
4815	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 7 reason = Timeout
4816	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
4817	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 13 reason = Timeout
4818	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
4819	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 13 reason = Timeout
4820	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
4821	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4822	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
4823	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4824	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
4825	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
4826	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
4827	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
4828	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4829	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
4830	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4831	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
4832	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4833	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4834	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4835	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4836	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
4837	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
4838	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4839	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
4840	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4841	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
4842	2024-12-06T21:20:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
4843	2024-12-06T21:20:54.408Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("709da000-e0da-4b29-8160-3112fd080927.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a03854a6-7a78-402b-93b4-65c6d36bf236.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a03854a6-7a78-402b-93b4-65c6d36bf236.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1169, tv_nsec: 906276988 } })
4844	2024-12-06T21:20:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
4845	2024-12-06T21:20:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
4846	2024-12-06T21:20:54.408Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4aff1ae3-2d71-40bf-bc46-79a7276da3c2.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4dccdb26-9cf9-40aa-9229-5d35905cf0e1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("79d475dc-44ec-44f4-9e51-ea8a753c39f3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("79d475dc-44ec-44f4-9e51-ea8a753c39f3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1169, tv_nsec: 906302128 } })
4847	2024-12-06T21:20:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
4848	2024-12-06T21:20:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4849	2024-12-06T21:20:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
4850	2024-12-06T21:20:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
4851	2024-12-06T21:20:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
4852	2024-12-06T21:20:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
4853	2024-12-06T21:20:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
4854	2024-12-06T21:20:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
4855	2024-12-06T21:20:54.408Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
4856	2024-12-06T21:20:54.408Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.134929ms iteration = 7 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
4857	2024-12-06T21:20:54.408Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
4858	2024-12-06T21:20:54.408Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.292741ms iteration = 7 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
4859	2024-12-06T21:20:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
4860	2024-12-06T21:20:54.409Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4861	2024-12-06T21:20:54.409Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4862	2024-12-06T21:20:54.410Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4863	2024-12-06T21:20:54.411Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.399942ms iteration = 13 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
4864	2024-12-06T21:20:54.412Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1169, tv_nsec: 911104427 } })
4865	2024-12-06T21:20:54.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
4866	2024-12-06T21:20:54.412Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
4867	2024-12-06T21:20:54.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
4868	2024-12-06T21:20:54.412Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4869	2024-12-06T21:20:54.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
4870	2024-12-06T21:20:54.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4871	2024-12-06T21:20:54.413Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
4872	2024-12-06T21:20:54.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
4873	2024-12-06T21:20:54.414Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4874	2024-12-06T21:20:54.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4875	2024-12-06T21:20:54.414Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4876	2024-12-06T21:20:54.414Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4877	2024-12-06T21:20:54.415Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4878	2024-12-06T21:20:54.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4879	2024-12-06T21:20:54.416Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
4880	2024-12-06T21:20:54.416Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4881	2024-12-06T21:20:54.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
4882	2024-12-06T21:20:54.416Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
4883	2024-12-06T21:20:54.417Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4884	2024-12-06T21:20:54.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4885	2024-12-06T21:20:54.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
4886	2024-12-06T21:20:54.417Z	DEBG	nexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
4887	2024-12-06T21:20:54.417Z	INFO	nexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
4888	2024-12-06T21:20:54.417Z	DEBG	nexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 10.623055ms iteration = 7 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
4889	2024-12-06T21:20:54.418Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4890	2024-12-06T21:20:54.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
4891	2024-12-06T21:20:54.418Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
4892	2024-12-06T21:20:54.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
4893	2024-12-06T21:20:54.419Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4894	2024-12-06T21:20:54.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4895	2024-12-06T21:20:54.419Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4896	2024-12-06T21:20:54.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4897	2024-12-06T21:20:54.421Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4898	2024-12-06T21:20:54.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4899	2024-12-06T21:20:54.421Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
4900	2024-12-06T21:20:54.421Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4901	2024-12-06T21:20:54.422Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4902	2024-12-06T21:20:54.422Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4903	2024-12-06T21:20:54.422Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4904	2024-12-06T21:20:54.424Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 17.202232ms iteration = 13 status = {"errors":[],"records_set_to_done":[]}
4905	2024-12-06T21:20:54.424Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1169, tv_nsec: 923284827 } })
4906	2024-12-06T21:20:54.425Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
4907	2024-12-06T21:20:54.426Z	DEBG	nexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 18.772638ms iteration = 7 status = {"errors":[],"found_port_ok":[]}
4908	2024-12-06T21:20:54.427Z	INFO	nexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-06 21:10:54.407304728 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
4909	2024-12-06T21:20:54.427Z	DEBG	nexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 20.046409ms iteration = 7 status = {"expiration":"2024-12-06T21:10:54.407304728Z","pruned":[]}
4910	2024-12-06T21:20:54.432Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4911	2024-12-06T21:20:54.433Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4912	2024-12-06T21:20:54.435Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
4913	2024-12-06T21:20:54.435Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
4914	2024-12-06T21:20:54.436Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
4915	2024-12-06T21:20:54.436Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4916	2024-12-06T21:20:54.436Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
4917	2024-12-06T21:20:54.436Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4918	2024-12-06T21:20:54.436Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4919	2024-12-06T21:20:54.437Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4920	2024-12-06T21:20:54.437Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4921	2024-12-06T21:20:54.437Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
4922	2024-12-06T21:20:54.437Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
4923	2024-12-06T21:20:54.438Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
4924	2024-12-06T21:20:54.439Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1169, tv_nsec: 938097710 } })
4925	2024-12-06T21:20:54.440Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
4926	2024-12-06T21:20:54.440Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4927	2024-12-06T21:20:54.441Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4928	2024-12-06T21:20:54.441Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4929	2024-12-06T21:20:54.442Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4930	2024-12-06T21:20:54.442Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4931	2024-12-06T21:20:54.443Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
4932	2024-12-06T21:20:54.443Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
4933	2024-12-06T21:20:54.443Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
4934	2024-12-06T21:20:54.443Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4935	2024-12-06T21:20:54.444Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 37.279492ms iteration = 37 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
4936	2024-12-06T21:20:54.445Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
4937	2024-12-06T21:20:54.445Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4938	2024-12-06T21:20:54.445Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4939	2024-12-06T21:20:54.446Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
4940	2024-12-06T21:20:54.446Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
4941	2024-12-06T21:20:54.446Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
4942	2024-12-06T21:20:54.446Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
4943	2024-12-06T21:20:54.447Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4944	2024-12-06T21:20:54.448Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4945	2024-12-06T21:20:54.448Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4946	2024-12-06T21:20:54.448Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4947	2024-12-06T21:20:54.449Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4948	2024-12-06T21:20:54.450Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4949	2024-12-06T21:20:54.450Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4950	2024-12-06T21:20:54.452Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
4951	2024-12-06T21:20:54.452Z	DEBG	nexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 45.71541ms iteration = 4 status = {"nsuccess":5}
4952	2024-12-06T21:20:54.453Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4953	2024-12-06T21:20:54.453Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4954	2024-12-06T21:20:54.453Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4955	2024-12-06T21:20:54.456Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
4956	2024-12-06T21:20:54.456Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4957	2024-12-06T21:20:54.456Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4958	2024-12-06T21:20:54.457Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
4959	2024-12-06T21:20:54.457Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4960	2024-12-06T21:20:54.457Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4961	2024-12-06T21:20:54.457Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
4962	2024-12-06T21:20:54.458Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4963	2024-12-06T21:20:54.458Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4964	2024-12-06T21:20:54.471Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4965	2024-12-06T21:20:54.471Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
4966	2024-12-06T21:20:54.471Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4967	2024-12-06T21:20:54.471Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
4968	2024-12-06T21:20:54.472Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
4969	2024-12-06T21:20:54.472Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
4970	2024-12-06T21:20:54.472Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
4971	2024-12-06T21:20:54.472Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
4972	2024-12-06T21:20:54.472Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
4973	2024-12-06T21:20:54.472Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
4974	2024-12-06T21:20:54.474Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4975	2024-12-06T21:20:54.474Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
4976	2024-12-06T21:20:54.474Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 67.038369ms iteration = 37 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
4977	2024-12-06T21:20:54.474Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4978	2024-12-06T21:20:54.474Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
4979	2024-12-06T21:20:54.474Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
4980	2024-12-06T21:20:54.475Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4981	2024-12-06T21:20:54.476Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
4982	2024-12-06T21:20:54.476Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
4983	2024-12-06T21:20:54.476Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
4984	2024-12-06T21:20:54.476Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 69.295856ms iteration = 13 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
4985	2024-12-06T21:20:54.476Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
4986	2024-12-06T21:20:54.477Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 69.888946ms iteration = 13 status = {"errors":[],"garbage_collect_requested":[]}
4987	2024-12-06T21:20:54.477Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cbc5c1c6-5668-455a-85ef-94f4aebe0f80, lookup_type: ById(cbc5c1c6-5668-455a-85ef-94f4aebe0f80) } result = Ok(())
4988	2024-12-06T21:20:54.477Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4989	2024-12-06T21:20:54.478Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4990	2024-12-06T21:20:54.480Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 73.678938ms iteration = 13 status = 0
4991	2024-12-06T21:20:54.482Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
4992	2024-12-06T21:20:54.482Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
4993	2024-12-06T21:20:54.482Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
4994	2024-12-06T21:20:54.482Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 75.784263ms iteration = 13 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
4995	2024-12-06T21:20:54.484Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 76.981552ms iteration = 13 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
4996	2024-12-06T21:20:54.490Z	DEBG	nexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
4997	2024-12-06T21:20:54.490Z	INFO	nexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
4998	2024-12-06T21:20:54.490Z	DEBG	nexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 82.998311ms iteration = 7 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
4999	2024-12-06T21:20:54.490Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
5000	2024-12-06T21:20:54.490Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
5001	2024-12-06T21:20:54.490Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
5002	2024-12-06T21:20:54.490Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
5003	2024-12-06T21:20:54.490Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
5004	2024-12-06T21:20:54.490Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5005	2024-12-06T21:20:54.490Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5006	2024-12-06T21:20:54.492Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5007	2024-12-06T21:20:54.493Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
5008	2024-12-06T21:20:54.493Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
5009	2024-12-06T21:20:54.493Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
5010	2024-12-06T21:20:54.493Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5011	2024-12-06T21:20:54.493Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5012	2024-12-06T21:20:54.495Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
5013	2024-12-06T21:20:54.495Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5014	2024-12-06T21:20:54.495Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5015	2024-12-06T21:20:54.495Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5016	2024-12-06T21:20:54.495Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5017	2024-12-06T21:20:54.496Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5018	2024-12-06T21:20:54.497Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5019	2024-12-06T21:20:54.498Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
5020	2024-12-06T21:20:54.498Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5021	2024-12-06T21:20:54.498Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5022	2024-12-06T21:20:54.500Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
5023	2024-12-06T21:20:54.500Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5024	2024-12-06T21:20:54.500Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5025	2024-12-06T21:20:54.501Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5026	2024-12-06T21:20:54.501Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5027	2024-12-06T21:20:54.503Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5028	2024-12-06T21:20:54.505Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
5029	2024-12-06T21:20:54.505Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5030	2024-12-06T21:20:54.505Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5031	2024-12-06T21:20:54.506Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
5032	2024-12-06T21:20:54.507Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5033	2024-12-06T21:20:54.509Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
5034	2024-12-06T21:20:54.509Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5035	2024-12-06T21:20:54.510Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5036	2024-12-06T21:20:54.514Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
5037	2024-12-06T21:20:54.514Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
5038	2024-12-06T21:20:54.514Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 106.967774ms iteration = 13 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
5039	2024-12-06T21:20:54.517Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 109.966693ms iteration = 13 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
5040	2024-12-06T21:20:54.537Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
5041	2024-12-06T21:20:54.537Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 93c3f865-bc00-42df-bef7-20e39342f84f dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
5042	2024-12-06T21:20:54.537Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5043	2024-12-06T21:20:54.537Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5044	2024-12-06T21:20:54.538Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5045	2024-12-06T21:20:54.539Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
5046	2024-12-06T21:20:54.539Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 05c53cbc-f66c-491f-af8d-6f291fe21596 dns_zone_name = oxide.test found_so_far = 0 version = 2
5047	2024-12-06T21:20:54.540Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5048	2024-12-06T21:20:54.540Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5049	2024-12-06T21:20:54.541Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
5050	2024-12-06T21:20:54.541Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5051	2024-12-06T21:20:54.541Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5052	2024-12-06T21:20:54.542Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5053	2024-12-06T21:20:54.544Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
5054	2024-12-06T21:20:54.544Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5055	2024-12-06T21:20:54.545Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5056	2024-12-06T21:20:54.546Z	DEBG	nexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-06T21:14:50.566199Z, time_modified: 2024-12-06T21:14:50.566199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, db_silo: Silo { identity: SiloIdentity { id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-06T21:15:06.991365Z, time_modified: 2024-12-06T21:15:06.991365Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, db_silo: Silo { identity: SiloIdentity { id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-06T21:15:06.991365Z, time_modified: 2024-12-06T21:15:06.991365Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1" }] }) }
5057	2024-12-06T21:20:54.546Z	DEBG	nexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 139.0064ms iteration = 7 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"683d02e1-6ec8-48bf-98d0-db6c94785f80","tls_certs":["d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1"]}},"default_endpoint":{"silo_id":"683d02e1-6ec8-48bf-98d0-db6c94785f80","tls_certs":["d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
5058	2024-12-06T21:20:54.548Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 93c3f865-bc00-42df-bef7-20e39342f84f dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
5059	2024-12-06T21:20:54.548Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
5060	2024-12-06T21:20:54.548Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-06 21:15:06.025861 UTC dns_group = internal generation = 1
5061	2024-12-06T21:20:54.548Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 141.286818ms iteration = 7 status = {"generation":1}
5062	2024-12-06T21:20:54.578Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 171.252789ms iteration = 14 status = {"physical_disks_added":0}
5063	2024-12-06T21:20:54.585Z	WARN	nexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
5064	2024-12-06T21:20:54.585Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 178.071861ms iteration = 7 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
5065	2024-12-06T21:20:54.595Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
5066	2024-12-06T21:20:54.595Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5067	2024-12-06T21:20:54.595Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "391f0901-eea3-41a0-b286-44e09a713158", "content-length": "26", "date": "Fri, 06 Dec 2024 21:20:54 GMT"} })
5068	2024-12-06T21:20:54.595Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5069	2024-12-06T21:20:54.597Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
5070	2024-12-06T21:20:54.621Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 zone_address = fd00:1122:3344:101::2
5071	2024-12-06T21:20:54.621Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5072	2024-12-06T21:20:54.622Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "aac00a5f-438a-4e7d-8420-a7e565c41e3e", "content-length": "26", "date": "Fri, 06 Dec 2024 21:20:54 GMT"} })
5073	2024-12-06T21:20:54.622Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5074	2024-12-06T21:20:54.624Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
5075	2024-12-06T21:20:54.624Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
5076	2024-12-06T21:20:54.738Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
5077	2024-12-06T21:20:54.738Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5078	2024-12-06T21:20:54.738Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6823922e-59bf-42f3-be64-365ed5cc34e5", "content-length": "26", "date": "Fri, 06 Dec 2024 21:20:54 GMT"} })
5079	2024-12-06T21:20:54.738Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5080	2024-12-06T21:20:54.739Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
5081	2024-12-06T21:20:54.739Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
5082	2024-12-06T21:20:54.760Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
5083	2024-12-06T21:20:54.760Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
5084	2024-12-06T21:20:54.760Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "0cb99c66-53f2-499a-b549-db79fe02a5cb", "content-length": "372", "date": "Fri, 06 Dec 2024 21:20:54 GMT"} })
5085	2024-12-06T21:20:54.760Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5086	2024-12-06T21:20:54.761Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5087	2024-12-06T21:20:54.763Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 356.65528ms iteration = 13 status = {}
5088	2024-12-06T21:20:54.785Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
5089	2024-12-06T21:20:54.786Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "ed42f2bd-81e3-4af9-a51c-efa1d208849f", "content-length": "691", "date": "Fri, 06 Dec 2024 21:20:54 GMT"} })
5090	2024-12-06T21:20:54.786Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
5091	2024-12-06T21:20:54.786Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
5092	2024-12-06T21:20:54.786Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
5093	2024-12-06T21:20:54.788Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 05c53cbc-f66c-491f-af8d-6f291fe21596 dns_zone_name = oxide.test found_so_far = 1 version = 2
5094	2024-12-06T21:20:54.788Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
5095	2024-12-06T21:20:54.788Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-06 21:15:07.045426 UTC dns_group = external generation = 2
5096	2024-12-06T21:20:54.788Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 380.782925ms iteration = 7 status = {"generation":2}
5097	2024-12-06T21:20:54.788Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 380.941099ms iteration = 13 status = {}
5098	2024-12-06T21:20:54.802Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
5099	2024-12-06T21:20:54.802Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
5100	2024-12-06T21:20:54.803Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "6bb01c5e-9511-45a7-8c67-dcdc7734965d", "date": "Fri, 06 Dec 2024 21:20:54 GMT"} })
5101	2024-12-06T21:20:54.803Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5102	2024-12-06T21:20:54.803Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5103	2024-12-06T21:20:54.806Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5104	2024-12-06T21:20:54.806Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5105	2024-12-06T21:20:54.809Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5106	2024-12-06T21:20:54.809Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5107	2024-12-06T21:20:54.813Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
5108	2024-12-06T21:20:54.813Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
5109	2024-12-06T21:20:54.813Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "b984f8af-33b5-40b4-a336-5b6a14ec3057", "date": "Fri, 06 Dec 2024 21:20:54 GMT"} })
5110	2024-12-06T21:20:54.815Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 408.131854ms iteration = 13 status = {}
5111	2024-12-06T21:20:54.839Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
5112	2024-12-06T21:20:54.839Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "17734b6e-8b08-4d9c-89c2-31281637c03a", "date": "Fri, 06 Dec 2024 21:20:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
5113	2024-12-06T21:20:54.839Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
5114	2024-12-06T21:20:54.841Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "f9de320f-e9c9-4558-a331-f1fc80cc0906", "date": "Fri, 06 Dec 2024 21:20:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
5115	2024-12-06T21:20:54.841Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
5116	2024-12-06T21:20:54.843Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
5117	2024-12-06T21:20:54.843Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 436.393608ms iteration = 9 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
5118	2024-12-06T21:20:54.853Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
5119	2024-12-06T21:20:54.854Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "c92d688a-3bbe-4379-800c-4846578f5f87", "content-length": "1", "date": "Fri, 06 Dec 2024 21:20:53 GMT"} })
5120	2024-12-06T21:20:54.855Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5121	2024-12-06T21:20:54.855Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5122	2024-12-06T21:20:54.858Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 450.903776ms iteration = 13 status = null
5123	2024-12-06T21:20:54.859Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
5124	2024-12-06T21:20:54.860Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "20c4062f-3914-426c-a502-e8bf16eb0709", "date": "Fri, 06 Dec 2024 21:20:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
5125	2024-12-06T21:20:54.860Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
5126	2024-12-06T21:20:54.861Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "d3dc0ee8-4b46-4c4a-9b01-7631a4e85352", "date": "Fri, 06 Dec 2024 21:20:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
5127	2024-12-06T21:20:54.861Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
5128	2024-12-06T21:20:54.863Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "28c68dbb-5a74-4256-8664-ab7fe417fde1", "date": "Fri, 06 Dec 2024 21:20:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
5129	2024-12-06T21:20:54.863Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
5130	2024-12-06T21:20:54.865Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
5131	2024-12-06T21:20:54.865Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 458.598862ms iteration = 9 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
5132	2024-12-06T21:20:54.905Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5133	2024-12-06T21:20:54.906Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5134	2024-12-06T21:20:54.907Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 3d56b7bf-53ed-4147-b45b-9cef154d7338, rack_id: 2d75e4a3-6363-4fd1-935e-989d31fd7d71, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(02859c9b-92d7-4b19-b425-8f266763be54) } rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
5135	2024-12-06T21:20:54.907Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5136	2024-12-06T21:20:54.908Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5137	2024-12-06T21:20:54.915Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
5138	2024-12-06T21:20:54.915Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "79ec6752-7991-4890-8830-d83da0acdbf8", "content-length": "134", "date": "Fri, 06 Dec 2024 21:20:53 GMT"} })
5139	2024-12-06T21:20:54.915Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
5140	2024-12-06T21:20:54.915Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 02859c9b-92d7-4b19-b425-8f266763be54
5141	2024-12-06T21:20:54.916Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
5142	2024-12-06T21:20:54.916Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
5143	2024-12-06T21:20:54.916Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5144	2024-12-06T21:20:54.916Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5145	2024-12-06T21:20:54.916Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5146	2024-12-06T21:20:54.916Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
5147	2024-12-06T21:20:54.916Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5148	2024-12-06T21:20:54.916Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
5149	2024-12-06T21:20:54.916Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
5150	2024-12-06T21:20:54.916Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5151	2024-12-06T21:20:54.916Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5152	2024-12-06T21:20:54.918Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
5153	2024-12-06T21:20:54.918Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "4ca9616e-09d9-4e63-ac16-295ea70e603c", "content-length": "2", "date": "Fri, 06 Dec 2024 21:20:53 GMT"} })
5154	2024-12-06T21:20:54.918Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
5155	2024-12-06T21:20:54.918Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ac123ad-9c83-40d6-8967-5c20c1f2e627", "content-length": "52", "date": "Fri, 06 Dec 2024 21:20:53 GMT"} })
5156	2024-12-06T21:20:54.918Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
5157	2024-12-06T21:20:54.918Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
5158	2024-12-06T21:20:54.981Z	DEBG	nexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
5159	2024-12-06T21:20:54.981Z	DEBG	nexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
5160	2024-12-06T21:20:55.012Z	DEBG	nexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
5161	2024-12-06T21:20:55.012Z	INFO	nexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:61485
5162	2024-12-06T21:20:55.012Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:61485 req_id = e0a08816-1133-4bab-b702-c1090c4579e5 roles = RoleSet { roles: {} } uri = /metrics/producers
5163	2024-12-06T21:20:55.012Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:61485 req_id = e0a08816-1133-4bab-b702-c1090c4579e5 resource = Database result = Ok(()) uri = /metrics/producers
5164	2024-12-06T21:20:55.015Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "f4d65e79-c9b9-4483-92b9-ce4b5c6fe68a", "date": "Fri, 06 Dec 2024 21:20:54 GMT"} })
5165	2024-12-06T21:20:55.015Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch = Switch0
5166	2024-12-06T21:20:55.015Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
5167	2024-12-06T21:20:55.015Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "251e5395-23be-4dd0-bdd9-8f6aacbb4ab4", "content-length": "682", "date": "Fri, 06 Dec 2024 21:20:54 GMT"} })
5168	2024-12-06T21:20:55.015Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5169	2024-12-06T21:20:55.016Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5170	2024-12-06T21:20:55.019Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5171	2024-12-06T21:20:55.020Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5172	2024-12-06T21:20:55.023Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5173	2024-12-06T21:20:55.023Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5174	2024-12-06T21:20:55.025Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5175	2024-12-06T21:20:55.025Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5176	2024-12-06T21:20:55.036Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 629.421654ms iteration = 13 status = {}
5177	2024-12-06T21:20:55.053Z	INFO	nexus: assigned collector to new producer collector_id = 9d509eff-9671-46fb-8274-6ec9019a9d9e file = nexus/src/app/oximeter.rs:76 producer_id = e8725c8c-13ab-4916-bf16-bb7f832014d5
5178	2024-12-06T21:20:55.053Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 41205 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:61485 req_id = e0a08816-1133-4bab-b702-c1090c4579e5 response_code = 201 uri = /metrics/producers
5179	2024-12-06T21:20:55.053Z	DEBG	nexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "e0a08816-1133-4bab-b702-c1090c4579e5", "content-length": "41", "date": "Fri, 06 Dec 2024 21:20:55 GMT"} })
5180	2024-12-06T21:20:55.054Z	DEBG	nexus (producer-registration-task): registered with nexus successfully
5181	2024-12-06T21:20:55.054Z	DEBG	nexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
5182	2024-12-06T21:20:56.541Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 283 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = cfe467d7-168d-4e84-92a0-0b2c61a5049c response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
5183	2024-12-06T21:21:04.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 38 reason = Timeout
5184	2024-12-06T21:21:04.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 38 reason = Timeout
5185	2024-12-06T21:21:04.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5186	2024-12-06T21:21:04.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5187	2024-12-06T21:21:04.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5188	2024-12-06T21:21:04.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5189	2024-12-06T21:21:04.408Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5190	2024-12-06T21:21:04.408Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5191	2024-12-06T21:21:04.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5192	2024-12-06T21:21:04.410Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.982345ms iteration = 38 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
5193	2024-12-06T21:21:04.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5194	2024-12-06T21:21:04.411Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5195	2024-12-06T21:21:04.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5196	2024-12-06T21:21:04.413Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5197	2024-12-06T21:21:04.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5198	2024-12-06T21:21:04.414Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5199	2024-12-06T21:21:04.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
5200	2024-12-06T21:21:04.417Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5201	2024-12-06T21:21:04.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5202	2024-12-06T21:21:04.426Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
5203	2024-12-06T21:21:04.426Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.150351ms iteration = 38 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
5204	2024-12-06T21:21:06.540Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 298 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = d5a14d96-ffd1-40ab-a35a-865313982d6a response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
5205	2024-12-06T21:21:14.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 39 reason = Timeout
5206	2024-12-06T21:21:14.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 39 reason = Timeout
5207	2024-12-06T21:21:14.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5208	2024-12-06T21:21:14.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5209	2024-12-06T21:21:14.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5210	2024-12-06T21:21:14.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5211	2024-12-06T21:21:14.408Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5212	2024-12-06T21:21:14.409Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5213	2024-12-06T21:21:14.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5214	2024-12-06T21:21:14.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5215	2024-12-06T21:21:14.411Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5216	2024-12-06T21:21:14.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5217	2024-12-06T21:21:14.411Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.718008ms iteration = 39 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
5218	2024-12-06T21:21:14.413Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5219	2024-12-06T21:21:14.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5220	2024-12-06T21:21:14.415Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5221	2024-12-06T21:21:14.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
5222	2024-12-06T21:21:14.417Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5223	2024-12-06T21:21:14.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5224	2024-12-06T21:21:14.425Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
5225	2024-12-06T21:21:14.425Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.864969ms iteration = 39 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
5226	2024-12-06T21:21:16.541Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 398 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 339e4ccc-a852-41db-9aac-c157851e32ca response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
5227	2024-12-06T21:21:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 14 reason = Timeout
5228	2024-12-06T21:21:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5229	2024-12-06T21:21:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 14 reason = Timeout
5230	2024-12-06T21:21:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
5231	2024-12-06T21:21:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 14 reason = Timeout
5232	2024-12-06T21:21:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 14 reason = Timeout
5233	2024-12-06T21:21:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5234	2024-12-06T21:21:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5235	2024-12-06T21:21:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 14 reason = Timeout
5236	2024-12-06T21:21:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 40 reason = Timeout
5237	2024-12-06T21:21:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 14 reason = Timeout
5238	2024-12-06T21:21:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 40 reason = Timeout
5239	2024-12-06T21:21:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5240	2024-12-06T21:21:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5241	2024-12-06T21:21:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 14 reason = Timeout
5242	2024-12-06T21:21:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5243	2024-12-06T21:21:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
5244	2024-12-06T21:21:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 14 reason = Timeout
5245	2024-12-06T21:21:24.406Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
5246	2024-12-06T21:21:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 14 reason = Timeout
5247	2024-12-06T21:21:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5248	2024-12-06T21:21:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
5249	2024-12-06T21:21:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 14 reason = Timeout
5250	2024-12-06T21:21:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
5251	2024-12-06T21:21:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 14 reason = Timeout
5252	2024-12-06T21:21:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5253	2024-12-06T21:21:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 15 reason = Timeout
5254	2024-12-06T21:21:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5255	2024-12-06T21:21:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 14 reason = Timeout
5256	2024-12-06T21:21:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
5257	2024-12-06T21:21:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 14 reason = Timeout
5258	2024-12-06T21:21:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5259	2024-12-06T21:21:24.406Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 14 reason = Timeout
5260	2024-12-06T21:21:24.406Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
5261	2024-12-06T21:21:24.406Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5262	2024-12-06T21:21:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5263	2024-12-06T21:21:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
5264	2024-12-06T21:21:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5265	2024-12-06T21:21:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5266	2024-12-06T21:21:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5267	2024-12-06T21:21:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5268	2024-12-06T21:21:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
5269	2024-12-06T21:21:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
5270	2024-12-06T21:21:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5271	2024-12-06T21:21:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5272	2024-12-06T21:21:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5273	2024-12-06T21:21:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
5274	2024-12-06T21:21:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5275	2024-12-06T21:21:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
5276	2024-12-06T21:21:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
5277	2024-12-06T21:21:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
5278	2024-12-06T21:21:24.410Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1199, tv_nsec: 908573520 } })
5279	2024-12-06T21:21:24.410Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.806793ms iteration = 14 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
5280	2024-12-06T21:21:24.410Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
5281	2024-12-06T21:21:24.410Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5282	2024-12-06T21:21:24.411Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5283	2024-12-06T21:21:24.411Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
5284	2024-12-06T21:21:24.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
5285	2024-12-06T21:21:24.412Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1199, tv_nsec: 910980690 } })
5286	2024-12-06T21:21:24.412Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
5287	2024-12-06T21:21:24.413Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
5288	2024-12-06T21:21:24.413Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5289	2024-12-06T21:21:24.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5290	2024-12-06T21:21:24.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
5291	2024-12-06T21:21:24.413Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5292	2024-12-06T21:21:24.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5293	2024-12-06T21:21:24.414Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
5294	2024-12-06T21:21:24.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
5295	2024-12-06T21:21:24.415Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5296	2024-12-06T21:21:24.415Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5297	2024-12-06T21:21:24.416Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
5298	2024-12-06T21:21:24.416Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
5299	2024-12-06T21:21:24.416Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5300	2024-12-06T21:21:24.416Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5301	2024-12-06T21:21:24.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
5302	2024-12-06T21:21:24.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5303	2024-12-06T21:21:24.418Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5304	2024-12-06T21:21:24.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5305	2024-12-06T21:21:24.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
5306	2024-12-06T21:21:24.418Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5307	2024-12-06T21:21:24.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5308	2024-12-06T21:21:24.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
5309	2024-12-06T21:21:24.419Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5310	2024-12-06T21:21:24.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5311	2024-12-06T21:21:24.419Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 12.945302ms iteration = 14 status = {"errors":[],"records_set_to_done":[]}
5312	2024-12-06T21:21:24.420Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5313	2024-12-06T21:21:24.420Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 13.720015ms iteration = 14 status = {"errors":[],"garbage_collect_requested":[]}
5314	2024-12-06T21:21:24.421Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5315	2024-12-06T21:21:24.421Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 14.64449ms iteration = 14 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
5316	2024-12-06T21:21:24.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
5317	2024-12-06T21:21:24.421Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5318	2024-12-06T21:21:24.422Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
5319	2024-12-06T21:21:24.422Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5320	2024-12-06T21:21:24.422Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5321	2024-12-06T21:21:24.422Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
5322	2024-12-06T21:21:24.422Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5323	2024-12-06T21:21:24.423Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5324	2024-12-06T21:21:24.424Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5325	2024-12-06T21:21:24.424Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5326	2024-12-06T21:21:24.425Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5327	2024-12-06T21:21:24.425Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5328	2024-12-06T21:21:24.425Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
5329	2024-12-06T21:21:24.425Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
5330	2024-12-06T21:21:24.425Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5331	2024-12-06T21:21:24.425Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5332	2024-12-06T21:21:24.426Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5333	2024-12-06T21:21:24.426Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
5334	2024-12-06T21:21:24.426Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1199, tv_nsec: 924924378 } })
5335	2024-12-06T21:21:24.426Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
5336	2024-12-06T21:21:24.427Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5337	2024-12-06T21:21:24.428Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5338	2024-12-06T21:21:24.428Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
5339	2024-12-06T21:21:24.428Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5340	2024-12-06T21:21:24.429Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5341	2024-12-06T21:21:24.430Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
5342	2024-12-06T21:21:24.430Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5343	2024-12-06T21:21:24.431Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5344	2024-12-06T21:21:24.431Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
5345	2024-12-06T21:21:24.431Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 24.961929ms iteration = 14 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
5346	2024-12-06T21:21:24.442Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5347	2024-12-06T21:21:24.442Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5348	2024-12-06T21:21:24.444Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5349	2024-12-06T21:21:24.444Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5350	2024-12-06T21:21:24.461Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 54.602276ms iteration = 14 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
5351	2024-12-06T21:21:24.463Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5352	2024-12-06T21:21:24.463Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5353	2024-12-06T21:21:24.470Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
5354	2024-12-06T21:21:24.470Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
5355	2024-12-06T21:21:24.470Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 63.506202ms iteration = 14 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
5356	2024-12-06T21:21:24.519Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
5357	2024-12-06T21:21:24.519Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5358	2024-12-06T21:21:24.520Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "37a53b96-be73-497c-8d99-608e68bce3c1", "content-length": "26", "date": "Fri, 06 Dec 2024 21:21:24 GMT"} })
5359	2024-12-06T21:21:24.520Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5360	2024-12-06T21:21:24.522Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
5361	2024-12-06T21:21:24.528Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5362	2024-12-06T21:21:24.529Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5363	2024-12-06T21:21:24.543Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 zone_address = fd00:1122:3344:101::2
5364	2024-12-06T21:21:24.543Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5365	2024-12-06T21:21:24.544Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f7f9210-8f53-46c7-933d-3604b72b2006", "content-length": "26", "date": "Fri, 06 Dec 2024 21:21:24 GMT"} })
5366	2024-12-06T21:21:24.544Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5367	2024-12-06T21:21:24.545Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
5368	2024-12-06T21:21:24.613Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
5369	2024-12-06T21:21:24.613Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5370	2024-12-06T21:21:24.613Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ea94be41-bf62-43aa-ac93-f3e1b27fbfc3", "content-length": "26", "date": "Fri, 06 Dec 2024 21:21:24 GMT"} })
5371	2024-12-06T21:21:24.613Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5372	2024-12-06T21:21:24.615Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
5373	2024-12-06T21:21:24.638Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
5374	2024-12-06T21:21:24.638Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
5375	2024-12-06T21:21:24.639Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "479f3a7a-d3ea-4cfd-885d-b2a94f2fbeec", "content-length": "372", "date": "Fri, 06 Dec 2024 21:21:23 GMT"} })
5376	2024-12-06T21:21:24.639Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5377	2024-12-06T21:21:24.639Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5378	2024-12-06T21:21:24.651Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
5379	2024-12-06T21:21:24.652Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "5dac2ad1-9f78-4b90-8f6c-4f96d37b0cba", "content-length": "1", "date": "Fri, 06 Dec 2024 21:21:24 GMT"} })
5380	2024-12-06T21:21:24.654Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5381	2024-12-06T21:21:24.654Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5382	2024-12-06T21:21:24.655Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5383	2024-12-06T21:21:24.655Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5384	2024-12-06T21:21:24.657Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 250.678242ms iteration = 14 status = null
5385	2024-12-06T21:21:24.657Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5386	2024-12-06T21:21:24.662Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cbc5c1c6-5668-455a-85ef-94f4aebe0f80, lookup_type: ById(cbc5c1c6-5668-455a-85ef-94f4aebe0f80) } result = Ok(())
5387	2024-12-06T21:21:24.662Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5388	2024-12-06T21:21:24.662Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5389	2024-12-06T21:21:24.669Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 262.888002ms iteration = 14 status = 0
5390	2024-12-06T21:21:24.674Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
5391	2024-12-06T21:21:24.674Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 268.26874ms iteration = 40 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
5392	2024-12-06T21:21:24.688Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5393	2024-12-06T21:21:24.688Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 281.63833ms iteration = 14 status = {}
5394	2024-12-06T21:21:24.688Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5395	2024-12-06T21:21:24.691Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 284.854073ms iteration = 40 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
5396	2024-12-06T21:21:24.697Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5397	2024-12-06T21:21:24.699Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
5398	2024-12-06T21:21:24.699Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5399	2024-12-06T21:21:24.700Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5400	2024-12-06T21:21:24.701Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5401	2024-12-06T21:21:24.704Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
5402	2024-12-06T21:21:24.704Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5403	2024-12-06T21:21:24.704Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5404	2024-12-06T21:21:24.709Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
5405	2024-12-06T21:21:24.709Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
5406	2024-12-06T21:21:24.710Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "ee5dd87f-de13-48e1-ace7-37b134480676", "date": "Fri, 06 Dec 2024 21:21:23 GMT"} })
5407	2024-12-06T21:21:24.710Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5408	2024-12-06T21:21:24.710Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5409	2024-12-06T21:21:24.713Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5410	2024-12-06T21:21:24.713Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5411	2024-12-06T21:21:24.718Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5412	2024-12-06T21:21:24.718Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5413	2024-12-06T21:21:24.722Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
5414	2024-12-06T21:21:24.722Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
5415	2024-12-06T21:21:24.722Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "4ebc94f0-495c-4e08-91ac-d53f180ddad7", "date": "Fri, 06 Dec 2024 21:21:23 GMT"} })
5416	2024-12-06T21:21:24.724Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 318.085497ms iteration = 14 status = {}
5417	2024-12-06T21:21:24.730Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 323.996224ms iteration = 14 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
5418	2024-12-06T21:21:24.754Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5419	2024-12-06T21:21:24.754Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5420	2024-12-06T21:21:24.757Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 3d56b7bf-53ed-4147-b45b-9cef154d7338, rack_id: 2d75e4a3-6363-4fd1-935e-989d31fd7d71, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(02859c9b-92d7-4b19-b425-8f266763be54) } rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
5421	2024-12-06T21:21:24.757Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5422	2024-12-06T21:21:24.757Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5423	2024-12-06T21:21:24.760Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 354.25485ms iteration = 15 status = {"physical_disks_added":0}
5424	2024-12-06T21:21:24.767Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
5425	2024-12-06T21:21:24.768Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "d66ec5fc-68ab-4cfc-894f-6b462f68d751", "content-length": "134", "date": "Fri, 06 Dec 2024 21:21:24 GMT"} })
5426	2024-12-06T21:21:24.768Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
5427	2024-12-06T21:21:24.768Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 02859c9b-92d7-4b19-b425-8f266763be54
5428	2024-12-06T21:21:24.768Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
5429	2024-12-06T21:21:24.768Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
5430	2024-12-06T21:21:24.768Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5431	2024-12-06T21:21:24.768Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5432	2024-12-06T21:21:24.768Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5433	2024-12-06T21:21:24.768Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
5434	2024-12-06T21:21:24.768Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5435	2024-12-06T21:21:24.768Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
5436	2024-12-06T21:21:24.768Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
5437	2024-12-06T21:21:24.768Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5438	2024-12-06T21:21:24.769Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5439	2024-12-06T21:21:24.771Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
5440	2024-12-06T21:21:24.771Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a6d792a-87bc-4b13-9f27-d3b08791d047", "content-length": "2", "date": "Fri, 06 Dec 2024 21:21:24 GMT"} })
5441	2024-12-06T21:21:24.771Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
5442	2024-12-06T21:21:24.771Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "e38fa383-c35c-478e-8805-6cc9b130c213", "content-length": "52", "date": "Fri, 06 Dec 2024 21:21:24 GMT"} })
5443	2024-12-06T21:21:24.771Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
5444	2024-12-06T21:21:24.771Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
5445	2024-12-06T21:21:24.803Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
5446	2024-12-06T21:21:24.865Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "79f72e4a-8e0f-4b1e-bbf6-e27ea9619485", "date": "Fri, 06 Dec 2024 21:21:24 GMT"} })
5447	2024-12-06T21:21:24.865Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch = Switch0
5448	2024-12-06T21:21:24.865Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
5449	2024-12-06T21:21:24.866Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "6101cbbf-6ac4-434b-9a77-1186ee5adaf8", "content-length": "691", "date": "Fri, 06 Dec 2024 21:21:24 GMT"} })
5450	2024-12-06T21:21:24.866Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
5451	2024-12-06T21:21:24.866Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
5452	2024-12-06T21:21:24.866Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
5453	2024-12-06T21:21:24.866Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "6bd9faa1-acc1-4b36-910c-f982fa008dc3", "content-length": "682", "date": "Fri, 06 Dec 2024 21:21:24 GMT"} })
5454	2024-12-06T21:21:24.866Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5455	2024-12-06T21:21:24.866Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5456	2024-12-06T21:21:24.867Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 461.074502ms iteration = 14 status = {}
5457	2024-12-06T21:21:24.870Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5458	2024-12-06T21:21:24.870Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5459	2024-12-06T21:21:24.874Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5460	2024-12-06T21:21:24.874Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5461	2024-12-06T21:21:24.876Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5462	2024-12-06T21:21:24.876Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5463	2024-12-06T21:21:24.883Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 476.740399ms iteration = 14 status = {}
5464	2024-12-06T21:21:26.541Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 469 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 12c04f72-16c3-46b3-a01f-10d98c464d65 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
5465	2024-12-06T21:21:34.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 41 reason = Timeout
5466	2024-12-06T21:21:34.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 41 reason = Timeout
5467	2024-12-06T21:21:34.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5468	2024-12-06T21:21:34.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5469	2024-12-06T21:21:34.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5470	2024-12-06T21:21:34.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5471	2024-12-06T21:21:34.408Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5472	2024-12-06T21:21:34.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5473	2024-12-06T21:21:34.411Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5474	2024-12-06T21:21:34.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5475	2024-12-06T21:21:34.413Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5476	2024-12-06T21:21:34.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5477	2024-12-06T21:21:34.415Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5478	2024-12-06T21:21:34.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
5479	2024-12-06T21:21:34.417Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5480	2024-12-06T21:21:34.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5481	2024-12-06T21:21:34.429Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
5482	2024-12-06T21:21:34.429Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.153993ms iteration = 41 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
5483	2024-12-06T21:21:34.436Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5484	2024-12-06T21:21:34.436Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5485	2024-12-06T21:21:34.464Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 57.630975ms iteration = 41 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
5486	2024-12-06T21:21:36.540Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 260 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = cacc0424-46df-4faa-937d-5bdd630eb232 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
5487	2024-12-06T21:21:44.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 42 reason = Timeout
5488	2024-12-06T21:21:44.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 42 reason = Timeout
5489	2024-12-06T21:21:44.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5490	2024-12-06T21:21:44.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5491	2024-12-06T21:21:44.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5492	2024-12-06T21:21:44.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5493	2024-12-06T21:21:44.408Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5494	2024-12-06T21:21:44.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5495	2024-12-06T21:21:44.409Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5496	2024-12-06T21:21:44.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5497	2024-12-06T21:21:44.411Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5498	2024-12-06T21:21:44.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5499	2024-12-06T21:21:44.414Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5500	2024-12-06T21:21:44.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5501	2024-12-06T21:21:44.418Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5502	2024-12-06T21:21:44.420Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
5503	2024-12-06T21:21:44.420Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5504	2024-12-06T21:21:44.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5505	2024-12-06T21:21:44.432Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
5506	2024-12-06T21:21:44.432Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.103452ms iteration = 42 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
5507	2024-12-06T21:21:44.443Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 36.102272ms iteration = 42 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
5508	2024-12-06T21:21:46.541Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 254 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 0df584d1-e4f6-4059-8d45-8ff9f7c0cf32 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
5509	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 10 reason = Timeout
5510	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 8 reason = Timeout
5511	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
5512	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 8 reason = Timeout
5513	2024-12-06T21:21:54.407Z	INFO	nexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
5514	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 15 reason = Timeout
5515	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
5516	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 15 reason = Timeout
5517	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5518	2024-12-06T21:21:54.407Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
5519	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 15 reason = Timeout
5520	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5521	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
5522	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 15 reason = Timeout
5523	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
5524	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5525	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 9 reason = Timeout
5526	2024-12-06T21:21:54.407Z	WARN	nexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0
5527	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 8 reason = Timeout
5528	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 16.161\u{b5}s iteration = 9 status = {"enabled":false,"target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0"}
5529	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 15 reason = Timeout
5530	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 43 reason = Timeout
5531	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5532	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 15 reason = Timeout
5533	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5534	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 8 reason = Timeout
5535	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5536	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
5537	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 15 reason = Timeout
5538	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
5539	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 15 reason = Timeout
5540	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5541	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 8 reason = Timeout
5542	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
5543	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5544	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 16 reason = Timeout
5545	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 15 reason = Timeout
5546	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5547	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5548	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 8 reason = Timeout
5549	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 15 reason = Timeout
5550	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 15 reason = Timeout
5551	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
5552	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
5553	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 10 reason = Timeout
5554	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = external_endpoints iteration = 8 reason = Timeout
5555	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5556	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 8 reason = Timeout
5557	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
5558	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 43 reason = Timeout
5559	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 8 reason = Timeout
5560	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5561	2024-12-06T21:21:54.407Z	INFO	nexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-06 21:11:54.407494323 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
5562	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
5563	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 15 reason = Timeout
5564	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 15 reason = Timeout
5565	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 15 reason = Timeout
5566	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5567	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
5568	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 8 reason = Timeout
5569	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
5570	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5571	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
5572	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
5573	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5574	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
5575	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5576	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5577	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
5578	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
5579	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5580	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5581	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
5582	2024-12-06T21:21:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
5583	2024-12-06T21:21:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5584	2024-12-06T21:21:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5585	2024-12-06T21:21:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5586	2024-12-06T21:21:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5587	2024-12-06T21:21:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
5588	2024-12-06T21:21:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
5589	2024-12-06T21:21:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
5590	2024-12-06T21:21:54.408Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("709da000-e0da-4b29-8160-3112fd080927.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a03854a6-7a78-402b-93b4-65c6d36bf236.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a03854a6-7a78-402b-93b4-65c6d36bf236.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1229, tv_nsec: 906240632 } })
5591	2024-12-06T21:21:54.408Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4aff1ae3-2d71-40bf-bc46-79a7276da3c2.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4dccdb26-9cf9-40aa-9229-5d35905cf0e1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("79d475dc-44ec-44f4-9e51-ea8a753c39f3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("79d475dc-44ec-44f4-9e51-ea8a753c39f3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1229, tv_nsec: 906250732 } })
5592	2024-12-06T21:21:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5593	2024-12-06T21:21:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5594	2024-12-06T21:21:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
5595	2024-12-06T21:21:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
5596	2024-12-06T21:21:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5597	2024-12-06T21:21:54.408Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
5598	2024-12-06T21:21:54.408Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.261161ms iteration = 8 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
5599	2024-12-06T21:21:54.408Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
5600	2024-12-06T21:21:54.408Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.240831ms iteration = 8 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
5601	2024-12-06T21:21:54.410Z	DEBG	nexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 3.281343ms iteration = 8 status = {"errors":[],"found_port_ok":[]}
5602	2024-12-06T21:21:54.410Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5603	2024-12-06T21:21:54.411Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.717891ms iteration = 15 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
5604	2024-12-06T21:21:54.412Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5605	2024-12-06T21:21:54.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
5606	2024-12-06T21:21:54.412Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
5607	2024-12-06T21:21:54.412Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
5608	2024-12-06T21:21:54.412Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
5609	2024-12-06T21:21:54.412Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5610	2024-12-06T21:21:54.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
5611	2024-12-06T21:21:54.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
5612	2024-12-06T21:21:54.414Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5613	2024-12-06T21:21:54.414Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
5614	2024-12-06T21:21:54.414Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5615	2024-12-06T21:21:54.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5616	2024-12-06T21:21:54.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5617	2024-12-06T21:21:54.417Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1229, tv_nsec: 915385892 } })
5618	2024-12-06T21:21:54.417Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
5619	2024-12-06T21:21:54.417Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5620	2024-12-06T21:21:54.418Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5621	2024-12-06T21:21:54.420Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
5622	2024-12-06T21:21:54.420Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5623	2024-12-06T21:21:54.420Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5624	2024-12-06T21:21:54.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5625	2024-12-06T21:21:54.421Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5626	2024-12-06T21:21:54.422Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5627	2024-12-06T21:21:54.422Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5628	2024-12-06T21:21:54.422Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5629	2024-12-06T21:21:54.422Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5630	2024-12-06T21:21:54.423Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5631	2024-12-06T21:21:54.425Z	INFO	nexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-06 21:11:54.407494323 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
5632	2024-12-06T21:21:54.425Z	DEBG	nexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 18.036456ms iteration = 8 status = {"expiration":"2024-12-06T21:11:54.407494323Z","pruned":[]}
5633	2024-12-06T21:21:54.431Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1229, tv_nsec: 930060633 } })
5634	2024-12-06T21:21:54.432Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
5635	2024-12-06T21:21:54.433Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5636	2024-12-06T21:21:54.433Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5637	2024-12-06T21:21:54.434Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5638	2024-12-06T21:21:54.434Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5639	2024-12-06T21:21:54.461Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
5640	2024-12-06T21:21:54.462Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
5641	2024-12-06T21:21:54.463Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5642	2024-12-06T21:21:54.464Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 57.533494ms iteration = 15 status = {"errors":[],"records_set_to_done":[]}
5643	2024-12-06T21:21:54.465Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5644	2024-12-06T21:21:54.465Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
5645	2024-12-06T21:21:54.465Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5646	2024-12-06T21:21:54.465Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5647	2024-12-06T21:21:54.466Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 58.670232ms iteration = 15 status = {"errors":[],"garbage_collect_requested":[]}
5648	2024-12-06T21:21:54.467Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
5649	2024-12-06T21:21:54.467Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5650	2024-12-06T21:21:54.467Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5651	2024-12-06T21:21:54.468Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
5652	2024-12-06T21:21:54.468Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
5653	2024-12-06T21:21:54.469Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5654	2024-12-06T21:21:54.469Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5655	2024-12-06T21:21:54.470Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5656	2024-12-06T21:21:54.471Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5657	2024-12-06T21:21:54.471Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1229, tv_nsec: 969909886 } })
5658	2024-12-06T21:21:54.472Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
5659	2024-12-06T21:21:54.472Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5660	2024-12-06T21:21:54.472Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5661	2024-12-06T21:21:54.473Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
5662	2024-12-06T21:21:54.473Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
5663	2024-12-06T21:21:54.473Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5664	2024-12-06T21:21:54.473Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
5665	2024-12-06T21:21:54.473Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
5666	2024-12-06T21:21:54.473Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5667	2024-12-06T21:21:54.473Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5668	2024-12-06T21:21:54.474Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5669	2024-12-06T21:21:54.474Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
5670	2024-12-06T21:21:54.474Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5671	2024-12-06T21:21:54.475Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5672	2024-12-06T21:21:54.475Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5673	2024-12-06T21:21:54.476Z	DEBG	nexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
5674	2024-12-06T21:21:54.476Z	INFO	nexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
5675	2024-12-06T21:21:54.476Z	DEBG	nexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 69.401548ms iteration = 8 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
5676	2024-12-06T21:21:54.476Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
5677	2024-12-06T21:21:54.476Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5678	2024-12-06T21:21:54.476Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5679	2024-12-06T21:21:54.477Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
5680	2024-12-06T21:21:54.477Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5681	2024-12-06T21:21:54.477Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
5682	2024-12-06T21:21:54.480Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5683	2024-12-06T21:21:54.484Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
5684	2024-12-06T21:21:54.484Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5685	2024-12-06T21:21:54.484Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5686	2024-12-06T21:21:54.485Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5687	2024-12-06T21:21:54.487Z	DEBG	nexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
5688	2024-12-06T21:21:54.488Z	INFO	nexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
5689	2024-12-06T21:21:54.488Z	DEBG	nexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 80.528181ms iteration = 8 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
5690	2024-12-06T21:21:54.488Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
5691	2024-12-06T21:21:54.488Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5692	2024-12-06T21:21:54.488Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5693	2024-12-06T21:21:54.492Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5694	2024-12-06T21:21:54.492Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5695	2024-12-06T21:21:54.493Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5696	2024-12-06T21:21:54.493Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
5697	2024-12-06T21:21:54.493Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 86.237505ms iteration = 15 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
5698	2024-12-06T21:21:54.493Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5699	2024-12-06T21:21:54.494Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5700	2024-12-06T21:21:54.495Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5701	2024-12-06T21:21:54.495Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5702	2024-12-06T21:21:54.497Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5703	2024-12-06T21:21:54.497Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
5704	2024-12-06T21:21:54.497Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5705	2024-12-06T21:21:54.497Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
5706	2024-12-06T21:21:54.497Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5707	2024-12-06T21:21:54.497Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
5708	2024-12-06T21:21:54.499Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
5709	2024-12-06T21:21:54.499Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 91.997129ms iteration = 43 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
5710	2024-12-06T21:21:54.500Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cbc5c1c6-5668-455a-85ef-94f4aebe0f80, lookup_type: ById(cbc5c1c6-5668-455a-85ef-94f4aebe0f80) } result = Ok(())
5711	2024-12-06T21:21:54.500Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5712	2024-12-06T21:21:54.500Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5713	2024-12-06T21:21:54.505Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 98.416844ms iteration = 15 status = 0
5714	2024-12-06T21:21:54.508Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5715	2024-12-06T21:21:54.509Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 102.669984ms iteration = 15 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
5716	2024-12-06T21:21:54.511Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
5717	2024-12-06T21:21:54.515Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5718	2024-12-06T21:21:54.515Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5719	2024-12-06T21:21:54.515Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 108.421958ms iteration = 43 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
5720	2024-12-06T21:21:54.515Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
5721	2024-12-06T21:21:54.515Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
5722	2024-12-06T21:21:54.515Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 108.608161ms iteration = 15 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
5723	2024-12-06T21:21:54.516Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5724	2024-12-06T21:21:54.518Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
5725	2024-12-06T21:21:54.518Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5726	2024-12-06T21:21:54.518Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5727	2024-12-06T21:21:54.519Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 112.017877ms iteration = 15 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
5728	2024-12-06T21:21:54.520Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5729	2024-12-06T21:21:54.520Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5730	2024-12-06T21:21:54.520Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
5731	2024-12-06T21:21:54.520Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
5732	2024-12-06T21:21:54.520Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
5733	2024-12-06T21:21:54.520Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5734	2024-12-06T21:21:54.520Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5735	2024-12-06T21:21:54.520Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5736	2024-12-06T21:21:54.522Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 115.006566ms iteration = 15 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
5737	2024-12-06T21:21:54.522Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
5738	2024-12-06T21:21:54.522Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5739	2024-12-06T21:21:54.522Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5740	2024-12-06T21:21:54.522Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5741	2024-12-06T21:21:54.525Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
5742	2024-12-06T21:21:54.525Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5743	2024-12-06T21:21:54.525Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5744	2024-12-06T21:21:54.526Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5745	2024-12-06T21:21:54.527Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5746	2024-12-06T21:21:54.527Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
5747	2024-12-06T21:21:54.527Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 93c3f865-bc00-42df-bef7-20e39342f84f dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
5748	2024-12-06T21:21:54.527Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5749	2024-12-06T21:21:54.527Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5750	2024-12-06T21:21:54.528Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5751	2024-12-06T21:21:54.528Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5752	2024-12-06T21:21:54.532Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
5753	2024-12-06T21:21:54.532Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5754	2024-12-06T21:21:54.532Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5755	2024-12-06T21:21:54.533Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
5756	2024-12-06T21:21:54.533Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5757	2024-12-06T21:21:54.533Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5758	2024-12-06T21:21:54.537Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 93c3f865-bc00-42df-bef7-20e39342f84f dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
5759	2024-12-06T21:21:54.537Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
5760	2024-12-06T21:21:54.537Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-06 21:15:06.025861 UTC dns_group = internal generation = 1
5761	2024-12-06T21:21:54.537Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 129.83816ms iteration = 8 status = {"generation":1}
5762	2024-12-06T21:21:54.542Z	WARN	nexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
5763	2024-12-06T21:21:54.542Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 135.3543ms iteration = 8 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
5764	2024-12-06T21:21:54.548Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
5765	2024-12-06T21:21:54.548Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
5766	2024-12-06T21:21:54.548Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
5767	2024-12-06T21:21:54.548Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5768	2024-12-06T21:21:54.548Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5769	2024-12-06T21:21:54.550Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5770	2024-12-06T21:21:54.552Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
5771	2024-12-06T21:21:54.552Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5772	2024-12-06T21:21:54.553Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5773	2024-12-06T21:21:54.554Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
5774	2024-12-06T21:21:54.554Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5775	2024-12-06T21:21:54.555Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "38dc551b-ecac-40b0-b2d0-5a32c27d965f", "content-length": "26", "date": "Fri, 06 Dec 2024 21:21:54 GMT"} })
5776	2024-12-06T21:21:54.555Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5777	2024-12-06T21:21:54.556Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
5778	2024-12-06T21:21:54.556Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
5779	2024-12-06T21:21:54.579Z	DEBG	nexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-06T21:14:50.566199Z, time_modified: 2024-12-06T21:14:50.566199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, db_silo: Silo { identity: SiloIdentity { id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-06T21:15:06.991365Z, time_modified: 2024-12-06T21:15:06.991365Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, db_silo: Silo { identity: SiloIdentity { id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-06T21:15:06.991365Z, time_modified: 2024-12-06T21:15:06.991365Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1" }] }) }
5780	2024-12-06T21:21:54.579Z	DEBG	nexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 171.706117ms iteration = 8 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"683d02e1-6ec8-48bf-98d0-db6c94785f80","tls_certs":["d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1"]}},"default_endpoint":{"silo_id":"683d02e1-6ec8-48bf-98d0-db6c94785f80","tls_certs":["d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
5781	2024-12-06T21:21:54.584Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
5782	2024-12-06T21:21:54.584Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 05c53cbc-f66c-491f-af8d-6f291fe21596 dns_zone_name = oxide.test found_so_far = 0 version = 2
5783	2024-12-06T21:21:54.584Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5784	2024-12-06T21:21:54.584Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5785	2024-12-06T21:21:54.586Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5786	2024-12-06T21:21:54.589Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
5787	2024-12-06T21:21:54.589Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5788	2024-12-06T21:21:54.589Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5789	2024-12-06T21:21:54.591Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 05c53cbc-f66c-491f-af8d-6f291fe21596 dns_zone_name = oxide.test found_so_far = 1 version = 2
5790	2024-12-06T21:21:54.591Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
5791	2024-12-06T21:21:54.591Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-06 21:15:07.045426 UTC dns_group = external generation = 2
5792	2024-12-06T21:21:54.591Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 183.809445ms iteration = 8 status = {"generation":2}
5793	2024-12-06T21:21:54.598Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 191.069914ms iteration = 16 status = {"physical_disks_added":0}
5794	2024-12-06T21:21:54.650Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
5795	2024-12-06T21:21:54.662Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 zone_address = fd00:1122:3344:101::2
5796	2024-12-06T21:21:54.662Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5797	2024-12-06T21:21:54.665Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a955dc27-51b9-4fbb-9b55-c42fe5f1c5d8", "content-length": "26", "date": "Fri, 06 Dec 2024 21:21:54 GMT"} })
5798	2024-12-06T21:21:54.665Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5799	2024-12-06T21:21:54.666Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
5800	2024-12-06T21:21:54.791Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
5801	2024-12-06T21:21:54.791Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5802	2024-12-06T21:21:54.791Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "276f5648-d79a-490e-91d3-43d761cda822", "content-length": "26", "date": "Fri, 06 Dec 2024 21:21:54 GMT"} })
5803	2024-12-06T21:21:54.791Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5804	2024-12-06T21:21:54.792Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
5805	2024-12-06T21:21:54.814Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
5806	2024-12-06T21:21:54.815Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "103f450b-53e5-457c-a69e-db73ab6dcb72", "date": "Fri, 06 Dec 2024 21:21:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
5807	2024-12-06T21:21:54.815Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
5808	2024-12-06T21:21:54.817Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "822d98c8-e6bc-482c-b991-292f052c973e", "date": "Fri, 06 Dec 2024 21:21:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
5809	2024-12-06T21:21:54.817Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
5810	2024-12-06T21:21:54.819Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
5811	2024-12-06T21:21:54.819Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 412.093009ms iteration = 10 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
5812	2024-12-06T21:21:54.837Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
5813	2024-12-06T21:21:54.837Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd455b9c-e39f-4bbf-804c-65d9f26fcdb0", "content-length": "691", "date": "Fri, 06 Dec 2024 21:21:53 GMT"} })
5814	2024-12-06T21:21:54.837Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
5815	2024-12-06T21:21:54.837Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
5816	2024-12-06T21:21:54.837Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
5817	2024-12-06T21:21:54.840Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 432.549955ms iteration = 15 status = {}
5818	2024-12-06T21:21:54.862Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 454.993252ms iteration = 15 status = {}
5819	2024-12-06T21:21:54.880Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
5820	2024-12-06T21:21:54.888Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
5821	2024-12-06T21:21:54.888Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
5822	2024-12-06T21:21:54.888Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "0ec1b91d-2d42-42fa-bbb2-6a7aa9fde785", "content-length": "372", "date": "Fri, 06 Dec 2024 21:21:53 GMT"} })
5823	2024-12-06T21:21:54.888Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5824	2024-12-06T21:21:54.888Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5825	2024-12-06T21:21:54.902Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
5826	2024-12-06T21:21:54.902Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
5827	2024-12-06T21:21:54.902Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "2b06f6b0-182d-4518-a9f2-5dd582b370ef", "date": "Fri, 06 Dec 2024 21:21:53 GMT"} })
5828	2024-12-06T21:21:54.902Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5829	2024-12-06T21:21:54.902Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5830	2024-12-06T21:21:54.905Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5831	2024-12-06T21:21:54.905Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5832	2024-12-06T21:21:54.908Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5833	2024-12-06T21:21:54.909Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5834	2024-12-06T21:21:54.911Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
5835	2024-12-06T21:21:54.911Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
5836	2024-12-06T21:21:54.912Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "a65314d9-ca6c-41cc-af3b-76a56e921828", "date": "Fri, 06 Dec 2024 21:21:53 GMT"} })
5837	2024-12-06T21:21:54.914Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 506.953605ms iteration = 15 status = {}
5838	2024-12-06T21:21:54.932Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
5839	2024-12-06T21:21:54.933Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "73666d05-4518-44f7-b821-dbf0d1e2cc2e", "content-length": "1", "date": "Fri, 06 Dec 2024 21:21:54 GMT"} })
5840	2024-12-06T21:21:54.934Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5841	2024-12-06T21:21:54.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5842	2024-12-06T21:21:54.937Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 529.943122ms iteration = 15 status = null
5843	2024-12-06T21:21:54.961Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
5844	2024-12-06T21:21:54.961Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "dd27fe60-b933-4734-8440-139a5788dcae", "date": "Fri, 06 Dec 2024 21:21:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
5845	2024-12-06T21:21:54.961Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
5846	2024-12-06T21:21:54.963Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "9c33a7bf-5159-4061-bba3-41b7e2298761", "date": "Fri, 06 Dec 2024 21:21:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
5847	2024-12-06T21:21:54.963Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
5848	2024-12-06T21:21:54.965Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "db8df19a-4a98-4d22-a362-a2d4280097d7", "date": "Fri, 06 Dec 2024 21:21:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
5849	2024-12-06T21:21:54.965Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
5850	2024-12-06T21:21:54.967Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
5851	2024-12-06T21:21:54.967Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 560.26801ms iteration = 10 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
5852	2024-12-06T21:21:54.981Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5853	2024-12-06T21:21:54.982Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5854	2024-12-06T21:21:54.984Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 3d56b7bf-53ed-4147-b45b-9cef154d7338, rack_id: 2d75e4a3-6363-4fd1-935e-989d31fd7d71, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(02859c9b-92d7-4b19-b425-8f266763be54) } rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
5855	2024-12-06T21:21:54.984Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5856	2024-12-06T21:21:54.984Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5857	2024-12-06T21:21:54.993Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
5858	2024-12-06T21:21:54.994Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8e3f123-f16e-4f56-b34a-46ca6a281be0", "content-length": "134", "date": "Fri, 06 Dec 2024 21:21:54 GMT"} })
5859	2024-12-06T21:21:54.994Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
5860	2024-12-06T21:21:54.994Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 02859c9b-92d7-4b19-b425-8f266763be54
5861	2024-12-06T21:21:54.994Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
5862	2024-12-06T21:21:54.994Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
5863	2024-12-06T21:21:54.994Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5864	2024-12-06T21:21:54.994Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5865	2024-12-06T21:21:54.994Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5866	2024-12-06T21:21:54.994Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
5867	2024-12-06T21:21:54.995Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5868	2024-12-06T21:21:54.995Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
5869	2024-12-06T21:21:54.995Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
5870	2024-12-06T21:21:54.995Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5871	2024-12-06T21:21:54.995Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5872	2024-12-06T21:21:54.997Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
5873	2024-12-06T21:21:54.997Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "a7f996fa-35f8-4d7a-8cb3-8e792eb5b564", "content-length": "2", "date": "Fri, 06 Dec 2024 21:21:54 GMT"} })
5874	2024-12-06T21:21:54.998Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
5875	2024-12-06T21:21:54.998Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "2de8bdd2-ae31-4357-9c08-0d8f6f011b97", "content-length": "52", "date": "Fri, 06 Dec 2024 21:21:54 GMT"} })
5876	2024-12-06T21:21:54.998Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
5877	2024-12-06T21:21:54.998Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
5878	2024-12-06T21:21:55.090Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "a9eebc3d-ee19-428f-8674-548882642901", "date": "Fri, 06 Dec 2024 21:21:54 GMT"} })
5879	2024-12-06T21:21:55.090Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch = Switch0
5880	2024-12-06T21:21:55.090Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
5881	2024-12-06T21:21:55.090Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "d54ae315-7124-43e8-982c-2057b8f888c3", "content-length": "682", "date": "Fri, 06 Dec 2024 21:21:55 GMT"} })
5882	2024-12-06T21:21:55.090Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5883	2024-12-06T21:21:55.091Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5884	2024-12-06T21:21:55.095Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5885	2024-12-06T21:21:55.095Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5886	2024-12-06T21:21:55.099Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5887	2024-12-06T21:21:55.099Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5888	2024-12-06T21:21:55.101Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5889	2024-12-06T21:21:55.101Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5890	2024-12-06T21:21:55.108Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 701.404354ms iteration = 15 status = {}
5891	2024-12-06T21:21:56.541Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 286 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = a7b9f839-d10f-4c94-af12-4278e8315de4 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
5892	2024-12-06T21:22:04.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 44 reason = Timeout
5893	2024-12-06T21:22:04.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 44 reason = Timeout
5894	2024-12-06T21:22:04.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5895	2024-12-06T21:22:04.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5896	2024-12-06T21:22:04.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5897	2024-12-06T21:22:04.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5898	2024-12-06T21:22:04.408Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5899	2024-12-06T21:22:04.408Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5900	2024-12-06T21:22:04.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5901	2024-12-06T21:22:04.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5902	2024-12-06T21:22:04.410Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5903	2024-12-06T21:22:04.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5904	2024-12-06T21:22:04.412Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5905	2024-12-06T21:22:04.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5906	2024-12-06T21:22:04.414Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5907	2024-12-06T21:22:04.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
5908	2024-12-06T21:22:04.416Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5909	2024-12-06T21:22:04.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5910	2024-12-06T21:22:04.427Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
5911	2024-12-06T21:22:04.427Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.034545ms iteration = 44 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
5912	2024-12-06T21:22:04.444Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 38.000813ms iteration = 44 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
5913	2024-12-06T21:22:06.541Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 261 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 4cf6e007-02d9-467b-948c-439e0431d521 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
5914	2024-12-06T21:22:14.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 45 reason = Timeout
5915	2024-12-06T21:22:14.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 45 reason = Timeout
5916	2024-12-06T21:22:14.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5917	2024-12-06T21:22:14.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5918	2024-12-06T21:22:14.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5919	2024-12-06T21:22:14.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5920	2024-12-06T21:22:14.409Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5921	2024-12-06T21:22:14.410Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5922	2024-12-06T21:22:14.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5923	2024-12-06T21:22:14.412Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.932071ms iteration = 45 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
5924	2024-12-06T21:22:14.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5925	2024-12-06T21:22:14.412Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5926	2024-12-06T21:22:14.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5927	2024-12-06T21:22:14.414Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5928	2024-12-06T21:22:14.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5929	2024-12-06T21:22:14.415Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5930	2024-12-06T21:22:14.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
5931	2024-12-06T21:22:14.418Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5932	2024-12-06T21:22:14.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5933	2024-12-06T21:22:14.430Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
5934	2024-12-06T21:22:14.430Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.431034ms iteration = 45 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
5935	2024-12-06T21:22:16.541Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 393 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = babdf6d7-1020-43a1-83f1-3e339d7a562a response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
5936	2024-12-06T21:22:21.534Z	INFO	nexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:54925
5937	2024-12-06T21:22:21.535Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54925 req_id = 536cfc59-6934-4bf7-b7b5-aa40de3f3655 roles = RoleSet { roles: {} } uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
5938	2024-12-06T21:22:21.535Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54925 req_id = 536cfc59-6934-4bf7-b7b5-aa40de3f3655 resource = Database result = Ok(()) uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
5939	2024-12-06T21:22:21.538Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3121 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54925 req_id = 536cfc59-6934-4bf7-b7b5-aa40de3f3655 response_code = 200 uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
5940	2024-12-06T21:22:21.538Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54925 req_id = f3a8f298-1470-4862-9316-57988938ef6f roles = RoleSet { roles: {} } uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
5941	2024-12-06T21:22:21.538Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54925 req_id = f3a8f298-1470-4862-9316-57988938ef6f resource = Database result = Ok(()) uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
5942	2024-12-06T21:22:21.540Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1933 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54925 req_id = f3a8f298-1470-4862-9316-57988938ef6f response_code = 200 uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
5943	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 16 reason = Timeout
5944	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5945	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 16 reason = Timeout
5946	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 16 reason = Timeout
5947	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
5948	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5949	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 16 reason = Timeout
5950	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 16 reason = Timeout
5951	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
5952	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5953	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 16 reason = Timeout
5954	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 46 reason = Timeout
5955	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5956	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 16 reason = Timeout
5957	2024-12-06T21:22:24.407Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
5958	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 17 reason = Timeout
5959	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5960	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 16 reason = Timeout
5961	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
5962	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 16 reason = Timeout
5963	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5964	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5965	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5966	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 16 reason = Timeout
5967	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
5968	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 16 reason = Timeout
5969	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
5970	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 16 reason = Timeout
5971	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
5972	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 16 reason = Timeout
5973	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5974	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 46 reason = Timeout
5975	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5976	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 16 reason = Timeout
5977	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5978	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5979	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
5980	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
5981	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5982	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5983	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5984	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
5985	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
5986	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5987	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
5988	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5989	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
5990	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5991	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
5992	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5993	2024-12-06T21:22:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5994	2024-12-06T21:22:24.409Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5995	2024-12-06T21:22:24.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
5996	2024-12-06T21:22:24.410Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5997	2024-12-06T21:22:24.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5998	2024-12-06T21:22:24.411Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.970835ms iteration = 16 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
5999	2024-12-06T21:22:24.411Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1259, tv_nsec: 909251745 } })
6000	2024-12-06T21:22:24.411Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6001	2024-12-06T21:22:24.411Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
6002	2024-12-06T21:22:24.412Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6003	2024-12-06T21:22:24.413Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
6004	2024-12-06T21:22:24.413Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6005	2024-12-06T21:22:24.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6006	2024-12-06T21:22:24.414Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1259, tv_nsec: 912733222 } })
6007	2024-12-06T21:22:24.414Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 7.465972ms iteration = 16 status = {"errors":[],"garbage_collect_requested":[]}
6008	2024-12-06T21:22:24.414Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
6009	2024-12-06T21:22:24.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6010	2024-12-06T21:22:24.415Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6011	2024-12-06T21:22:24.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6012	2024-12-06T21:22:24.415Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
6013	2024-12-06T21:22:24.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
6014	2024-12-06T21:22:24.416Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6015	2024-12-06T21:22:24.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6016	2024-12-06T21:22:24.417Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6017	2024-12-06T21:22:24.418Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6018	2024-12-06T21:22:24.419Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6019	2024-12-06T21:22:24.419Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1259, tv_nsec: 917997799 } })
6020	2024-12-06T21:22:24.420Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
6021	2024-12-06T21:22:24.420Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
6022	2024-12-06T21:22:24.420Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6023	2024-12-06T21:22:24.420Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6024	2024-12-06T21:22:24.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
6025	2024-12-06T21:22:24.421Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6026	2024-12-06T21:22:24.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6027	2024-12-06T21:22:24.421Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
6028	2024-12-06T21:22:24.421Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6029	2024-12-06T21:22:24.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6030	2024-12-06T21:22:24.423Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6031	2024-12-06T21:22:24.423Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6032	2024-12-06T21:22:24.425Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6033	2024-12-06T21:22:24.425Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6034	2024-12-06T21:22:24.427Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 20.13081ms iteration = 16 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
6035	2024-12-06T21:22:24.427Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6036	2024-12-06T21:22:24.428Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6037	2024-12-06T21:22:24.428Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6038	2024-12-06T21:22:24.429Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6039	2024-12-06T21:22:24.430Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6040	2024-12-06T21:22:24.430Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6041	2024-12-06T21:22:24.432Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
6042	2024-12-06T21:22:24.432Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6043	2024-12-06T21:22:24.432Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
6044	2024-12-06T21:22:24.432Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6045	2024-12-06T21:22:24.432Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6046	2024-12-06T21:22:24.432Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6047	2024-12-06T21:22:24.435Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 28.778842ms iteration = 17 status = {"physical_disks_added":0}
6048	2024-12-06T21:22:24.445Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
6049	2024-12-06T21:22:24.445Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 38.012754ms iteration = 46 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
6050	2024-12-06T21:22:24.447Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6051	2024-12-06T21:22:24.447Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
6052	2024-12-06T21:22:24.447Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6053	2024-12-06T21:22:24.448Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6054	2024-12-06T21:22:24.448Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
6055	2024-12-06T21:22:24.449Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
6056	2024-12-06T21:22:24.451Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6057	2024-12-06T21:22:24.451Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6058	2024-12-06T21:22:24.452Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 45.354244ms iteration = 16 status = {"errors":[],"records_set_to_done":[]}
6059	2024-12-06T21:22:24.453Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6060	2024-12-06T21:22:24.453Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6061	2024-12-06T21:22:24.455Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6062	2024-12-06T21:22:24.456Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6063	2024-12-06T21:22:24.457Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6064	2024-12-06T21:22:24.457Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6065	2024-12-06T21:22:24.458Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6066	2024-12-06T21:22:24.458Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 51.685427ms iteration = 16 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
6067	2024-12-06T21:22:24.462Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
6068	2024-12-06T21:22:24.462Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
6069	2024-12-06T21:22:24.462Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
6070	2024-12-06T21:22:24.465Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 58.396368ms iteration = 46 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6071	2024-12-06T21:22:24.485Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 78.377616ms iteration = 16 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
6072	2024-12-06T21:22:24.488Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6073	2024-12-06T21:22:24.488Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
6074	2024-12-06T21:22:24.488Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6075	2024-12-06T21:22:24.488Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6076	2024-12-06T21:22:24.491Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6077	2024-12-06T21:22:24.494Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cbc5c1c6-5668-455a-85ef-94f4aebe0f80, lookup_type: ById(cbc5c1c6-5668-455a-85ef-94f4aebe0f80) } result = Ok(())
6078	2024-12-06T21:22:24.494Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6079	2024-12-06T21:22:24.494Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6080	2024-12-06T21:22:24.496Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 89.180603ms iteration = 16 status = 0
6081	2024-12-06T21:22:24.505Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
6082	2024-12-06T21:22:24.505Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6083	2024-12-06T21:22:24.505Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "393709e0-dd1e-4c33-af94-4fdb7fad0486", "content-length": "26", "date": "Fri, 06 Dec 2024 21:22:24 GMT"} })
6084	2024-12-06T21:22:24.505Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6085	2024-12-06T21:22:24.507Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
6086	2024-12-06T21:22:24.507Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 100.200943ms iteration = 16 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
6087	2024-12-06T21:22:24.508Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
6088	2024-12-06T21:22:24.516Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 zone_address = fd00:1122:3344:101::2
6089	2024-12-06T21:22:24.516Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6090	2024-12-06T21:22:24.516Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f29346df-a97e-41b5-8816-df92f920ad01", "content-length": "26", "date": "Fri, 06 Dec 2024 21:22:24 GMT"} })
6091	2024-12-06T21:22:24.516Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6092	2024-12-06T21:22:24.518Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
6093	2024-12-06T21:22:24.520Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6094	2024-12-06T21:22:24.520Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
6095	2024-12-06T21:22:24.520Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 113.455701ms iteration = 16 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
6096	2024-12-06T21:22:24.521Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
6097	2024-12-06T21:22:24.521Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6098	2024-12-06T21:22:24.522Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "498fe95c-5679-498f-a87c-e5145a67f1b4", "content-length": "26", "date": "Fri, 06 Dec 2024 21:22:24 GMT"} })
6099	2024-12-06T21:22:24.522Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6100	2024-12-06T21:22:24.523Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
6101	2024-12-06T21:22:24.542Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6102	2024-12-06T21:22:24.543Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6103	2024-12-06T21:22:24.544Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6104	2024-12-06T21:22:24.546Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
6105	2024-12-06T21:22:24.546Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6106	2024-12-06T21:22:24.546Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6107	2024-12-06T21:22:24.547Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6108	2024-12-06T21:22:24.549Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
6109	2024-12-06T21:22:24.549Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6110	2024-12-06T21:22:24.549Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6111	2024-12-06T21:22:24.615Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
6112	2024-12-06T21:22:24.615Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
6113	2024-12-06T21:22:24.616Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "4d73d674-fdd7-4aec-a41f-bbabe67b6fe6", "content-length": "372", "date": "Fri, 06 Dec 2024 21:22:23 GMT"} })
6114	2024-12-06T21:22:24.616Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6115	2024-12-06T21:22:24.616Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6116	2024-12-06T21:22:24.629Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 222.201044ms iteration = 16 status = {}
6117	2024-12-06T21:22:24.643Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
6118	2024-12-06T21:22:24.643Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec6cb6e9-82b6-48bd-b1ed-3ecead591ad5", "content-length": "1", "date": "Fri, 06 Dec 2024 21:22:23 GMT"} })
6119	2024-12-06T21:22:24.645Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
6120	2024-12-06T21:22:24.645Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6121	2024-12-06T21:22:24.647Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 240.502374ms iteration = 16 status = null
6122	2024-12-06T21:22:24.670Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
6123	2024-12-06T21:22:24.670Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
6124	2024-12-06T21:22:24.670Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "198bc197-9bdd-4ccc-b62a-afde1e19a668", "date": "Fri, 06 Dec 2024 21:22:23 GMT"} })
6125	2024-12-06T21:22:24.670Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6126	2024-12-06T21:22:24.670Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6127	2024-12-06T21:22:24.673Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6128	2024-12-06T21:22:24.673Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6129	2024-12-06T21:22:24.681Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
6130	2024-12-06T21:22:24.681Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "050127e7-92cd-4e4e-836a-ec2794e651e0", "content-length": "691", "date": "Fri, 06 Dec 2024 21:22:23 GMT"} })
6131	2024-12-06T21:22:24.681Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
6132	2024-12-06T21:22:24.681Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
6133	2024-12-06T21:22:24.681Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
6134	2024-12-06T21:22:24.682Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 275.648641ms iteration = 16 status = {}
6135	2024-12-06T21:22:24.686Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6136	2024-12-06T21:22:24.686Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6137	2024-12-06T21:22:24.690Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
6138	2024-12-06T21:22:24.690Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
6139	2024-12-06T21:22:24.690Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "298fa6e4-1e7d-4bdb-ab96-14a95ed655a5", "date": "Fri, 06 Dec 2024 21:22:23 GMT"} })
6140	2024-12-06T21:22:24.692Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 285.500733ms iteration = 16 status = {}
6141	2024-12-06T21:22:24.735Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6142	2024-12-06T21:22:24.735Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6143	2024-12-06T21:22:24.737Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 3d56b7bf-53ed-4147-b45b-9cef154d7338, rack_id: 2d75e4a3-6363-4fd1-935e-989d31fd7d71, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(02859c9b-92d7-4b19-b425-8f266763be54) } rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
6144	2024-12-06T21:22:24.737Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6145	2024-12-06T21:22:24.737Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6146	2024-12-06T21:22:24.745Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
6147	2024-12-06T21:22:24.746Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "245df147-3680-410f-a10b-b14b18c26d06", "content-length": "134", "date": "Fri, 06 Dec 2024 21:22:24 GMT"} })
6148	2024-12-06T21:22:24.746Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
6149	2024-12-06T21:22:24.746Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 02859c9b-92d7-4b19-b425-8f266763be54
6150	2024-12-06T21:22:24.746Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
6151	2024-12-06T21:22:24.746Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
6152	2024-12-06T21:22:24.746Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
6153	2024-12-06T21:22:24.746Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
6154	2024-12-06T21:22:24.746Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
6155	2024-12-06T21:22:24.746Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
6156	2024-12-06T21:22:24.746Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
6157	2024-12-06T21:22:24.746Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
6158	2024-12-06T21:22:24.746Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
6159	2024-12-06T21:22:24.746Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6160	2024-12-06T21:22:24.747Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6161	2024-12-06T21:22:24.749Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
6162	2024-12-06T21:22:24.749Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "11adda62-4a82-4078-b69c-6f5e04881631", "content-length": "2", "date": "Fri, 06 Dec 2024 21:22:24 GMT"} })
6163	2024-12-06T21:22:24.749Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
6164	2024-12-06T21:22:24.749Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "00af6262-9fd2-4323-8d37-1777cea4de62", "content-length": "52", "date": "Fri, 06 Dec 2024 21:22:24 GMT"} })
6165	2024-12-06T21:22:24.749Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
6166	2024-12-06T21:22:24.749Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
6167	2024-12-06T21:22:24.840Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "e7b35ee0-a6de-4c47-a90e-998b8030f028", "date": "Fri, 06 Dec 2024 21:22:24 GMT"} })
6168	2024-12-06T21:22:24.840Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch = Switch0
6169	2024-12-06T21:22:24.840Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
6170	2024-12-06T21:22:24.841Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "d306dbfa-6cc8-4be9-b1b9-066656419e16", "content-length": "682", "date": "Fri, 06 Dec 2024 21:22:24 GMT"} })
6171	2024-12-06T21:22:24.841Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6172	2024-12-06T21:22:24.841Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6173	2024-12-06T21:22:24.845Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6174	2024-12-06T21:22:24.845Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6175	2024-12-06T21:22:24.848Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6176	2024-12-06T21:22:24.848Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6177	2024-12-06T21:22:24.850Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6178	2024-12-06T21:22:24.850Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6179	2024-12-06T21:22:24.857Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 449.958741ms iteration = 16 status = {}
6180	2024-12-06T21:22:26.541Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 273 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 01c0c2ab-eb76-42d2-9fb8-a536ee88574b response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
6181	2024-12-06T21:22:34.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 47 reason = Timeout
6182	2024-12-06T21:22:34.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 47 reason = Timeout
6183	2024-12-06T21:22:34.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6184	2024-12-06T21:22:34.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6185	2024-12-06T21:22:34.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6186	2024-12-06T21:22:34.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6187	2024-12-06T21:22:34.409Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6188	2024-12-06T21:22:34.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6189	2024-12-06T21:22:34.411Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6190	2024-12-06T21:22:34.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6191	2024-12-06T21:22:34.413Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6192	2024-12-06T21:22:34.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6193	2024-12-06T21:22:34.415Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6194	2024-12-06T21:22:34.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
6195	2024-12-06T21:22:34.417Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6196	2024-12-06T21:22:34.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6197	2024-12-06T21:22:34.427Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
6198	2024-12-06T21:22:34.427Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.782211ms iteration = 47 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
6199	2024-12-06T21:22:34.436Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6200	2024-12-06T21:22:34.437Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6201	2024-12-06T21:22:34.470Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 64.132122ms iteration = 47 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6202	2024-12-06T21:22:36.541Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 275 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = aab6515d-af3f-49c9-bdcf-f483c8d1e02f response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
6203	2024-12-06T21:22:44.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 48 reason = Timeout
6204	2024-12-06T21:22:44.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 48 reason = Timeout
6205	2024-12-06T21:22:44.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6206	2024-12-06T21:22:44.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6207	2024-12-06T21:22:44.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6208	2024-12-06T21:22:44.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6209	2024-12-06T21:22:44.409Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6210	2024-12-06T21:22:44.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6211	2024-12-06T21:22:44.412Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6212	2024-12-06T21:22:44.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6213	2024-12-06T21:22:44.414Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6214	2024-12-06T21:22:44.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6215	2024-12-06T21:22:44.416Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6216	2024-12-06T21:22:44.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
6217	2024-12-06T21:22:44.419Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6218	2024-12-06T21:22:44.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6219	2024-12-06T21:22:44.431Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
6220	2024-12-06T21:22:44.431Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.636387ms iteration = 48 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
6221	2024-12-06T21:22:44.437Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6222	2024-12-06T21:22:44.437Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6223	2024-12-06T21:22:44.438Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 30.984218ms iteration = 48 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6224	2024-12-06T21:22:46.541Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 429 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 931c533b-07e5-4834-9db6-824c7d464245 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
6225	2024-12-06T21:22:51.534Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54925 req_id = 166517bb-0533-4f41-ab34-e5974ffce5f8 roles = RoleSet { roles: {} } uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
6226	2024-12-06T21:22:51.535Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54925 req_id = 166517bb-0533-4f41-ab34-e5974ffce5f8 resource = Database result = Ok(()) uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
6227	2024-12-06T21:22:51.537Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2911 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54925 req_id = 166517bb-0533-4f41-ab34-e5974ffce5f8 response_code = 200 uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
6228	2024-12-06T21:22:51.537Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54925 req_id = 677ab891-046f-45fe-90a5-400e620d45c0 roles = RoleSet { roles: {} } uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
6229	2024-12-06T21:22:51.538Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54925 req_id = 677ab891-046f-45fe-90a5-400e620d45c0 resource = Database result = Ok(()) uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
6230	2024-12-06T21:22:51.540Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2559 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54925 req_id = 677ab891-046f-45fe-90a5-400e620d45c0 response_code = 200 uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
6231	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 49 reason = Timeout
6232	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6233	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 18 reason = Timeout
6234	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 17 reason = Timeout
6235	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6236	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6237	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 9 reason = Timeout
6238	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 11 reason = Timeout
6239	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
6240	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 17 reason = Timeout
6241	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 17 reason = Timeout
6242	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
6243	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 9 reason = Timeout
6244	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6245	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
6246	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 9 reason = Timeout
6247	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 17 reason = Timeout
6248	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
6249	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 9 reason = Timeout
6250	2024-12-06T21:22:54.407Z	INFO	nexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
6251	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
6252	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 17 reason = Timeout
6253	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
6254	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6255	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 17 reason = Timeout
6256	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
6257	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 17 reason = Timeout
6258	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 17 reason = Timeout
6259	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
6260	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6261	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 9 reason = Timeout
6262	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 9 reason = Timeout
6263	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
6264	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6265	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 10 reason = Timeout
6266	2024-12-06T21:22:54.407Z	WARN	nexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0
6267	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 12.75\u{b5}s iteration = 10 status = {"enabled":false,"target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0"}
6268	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 17 reason = Timeout
6269	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 9 reason = Timeout
6270	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 9 reason = Timeout
6271	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
6272	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
6273	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6274	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 17 reason = Timeout
6275	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
6276	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 17 reason = Timeout
6277	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 17 reason = Timeout
6278	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
6279	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
6280	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 11 reason = Timeout
6281	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 17 reason = Timeout
6282	2024-12-06T21:22:54.407Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
6283	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 17 reason = Timeout
6284	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6285	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 9 reason = Timeout
6286	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6287	2024-12-06T21:22:54.407Z	INFO	nexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-06 21:12:54.407623863 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
6288	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
6289	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 49 reason = Timeout
6290	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6291	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = external_endpoints iteration = 9 reason = Timeout
6292	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6293	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6294	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6295	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6296	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
6297	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
6298	2024-12-06T21:22:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6299	2024-12-06T21:22:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
6300	2024-12-06T21:22:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6301	2024-12-06T21:22:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
6302	2024-12-06T21:22:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
6303	2024-12-06T21:22:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
6304	2024-12-06T21:22:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6305	2024-12-06T21:22:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6306	2024-12-06T21:22:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
6307	2024-12-06T21:22:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
6308	2024-12-06T21:22:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6309	2024-12-06T21:22:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
6310	2024-12-06T21:22:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
6311	2024-12-06T21:22:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6312	2024-12-06T21:22:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6313	2024-12-06T21:22:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6314	2024-12-06T21:22:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6315	2024-12-06T21:22:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6316	2024-12-06T21:22:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
6317	2024-12-06T21:22:54.408Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("709da000-e0da-4b29-8160-3112fd080927.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a03854a6-7a78-402b-93b4-65c6d36bf236.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a03854a6-7a78-402b-93b4-65c6d36bf236.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1289, tv_nsec: 906250422 } })
6318	2024-12-06T21:22:54.408Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4aff1ae3-2d71-40bf-bc46-79a7276da3c2.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4dccdb26-9cf9-40aa-9229-5d35905cf0e1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("79d475dc-44ec-44f4-9e51-ea8a753c39f3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("79d475dc-44ec-44f4-9e51-ea8a753c39f3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1289, tv_nsec: 906250062 } })
6319	2024-12-06T21:22:54.408Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
6320	2024-12-06T21:22:54.408Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.350753ms iteration = 9 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
6321	2024-12-06T21:22:54.408Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
6322	2024-12-06T21:22:54.408Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.378893ms iteration = 9 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
6323	2024-12-06T21:22:54.410Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6324	2024-12-06T21:22:54.410Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.886457ms iteration = 17 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
6325	2024-12-06T21:22:54.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6326	2024-12-06T21:22:54.411Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6327	2024-12-06T21:22:54.412Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1289, tv_nsec: 910149086 } })
6328	2024-12-06T21:22:54.412Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
6329	2024-12-06T21:22:54.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
6330	2024-12-06T21:22:54.413Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6331	2024-12-06T21:22:54.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6332	2024-12-06T21:22:54.413Z	DEBG	nexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
6333	2024-12-06T21:22:54.413Z	INFO	nexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
6334	2024-12-06T21:22:54.413Z	DEBG	nexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 6.598559ms iteration = 9 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
6335	2024-12-06T21:22:54.415Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 8.024362ms iteration = 17 status = {"errors":[],"records_set_to_done":[]}
6336	2024-12-06T21:22:54.416Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6337	2024-12-06T21:22:54.416Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 8.54409ms iteration = 17 status = {"errors":[],"garbage_collect_requested":[]}
6338	2024-12-06T21:22:54.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6339	2024-12-06T21:22:54.417Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6340	2024-12-06T21:22:54.418Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
6341	2024-12-06T21:22:54.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
6342	2024-12-06T21:22:54.418Z	DEBG	nexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 10.851828ms iteration = 9 status = {"errors":[],"found_port_ok":[]}
6343	2024-12-06T21:22:54.418Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6344	2024-12-06T21:22:54.420Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6345	2024-12-06T21:22:54.420Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
6346	2024-12-06T21:22:54.420Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
6347	2024-12-06T21:22:54.420Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
6348	2024-12-06T21:22:54.421Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6349	2024-12-06T21:22:54.421Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
6350	2024-12-06T21:22:54.421Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6351	2024-12-06T21:22:54.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6352	2024-12-06T21:22:54.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6353	2024-12-06T21:22:54.422Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
6354	2024-12-06T21:22:54.422Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6355	2024-12-06T21:22:54.422Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6356	2024-12-06T21:22:54.422Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1289, tv_nsec: 920765700 } })
6357	2024-12-06T21:22:54.423Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
6358	2024-12-06T21:22:54.423Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6359	2024-12-06T21:22:54.424Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6360	2024-12-06T21:22:54.426Z	INFO	nexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-06 21:12:54.407623863 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
6361	2024-12-06T21:22:54.426Z	DEBG	nexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 18.899559ms iteration = 9 status = {"expiration":"2024-12-06T21:12:54.407623863Z","pruned":[]}
6362	2024-12-06T21:22:54.427Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6363	2024-12-06T21:22:54.427Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6364	2024-12-06T21:22:54.429Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1289, tv_nsec: 927781985 } })
6365	2024-12-06T21:22:54.430Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
6366	2024-12-06T21:22:54.470Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6367	2024-12-06T21:22:54.470Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6368	2024-12-06T21:22:54.472Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6369	2024-12-06T21:22:54.473Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6370	2024-12-06T21:22:54.474Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
6371	2024-12-06T21:22:54.474Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
6372	2024-12-06T21:22:54.475Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
6373	2024-12-06T21:22:54.475Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6374	2024-12-06T21:22:54.475Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6375	2024-12-06T21:22:54.477Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
6376	2024-12-06T21:22:54.477Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6377	2024-12-06T21:22:54.477Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
6378	2024-12-06T21:22:54.477Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6379	2024-12-06T21:22:54.477Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6380	2024-12-06T21:22:54.478Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6381	2024-12-06T21:22:54.478Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6382	2024-12-06T21:22:54.478Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6383	2024-12-06T21:22:54.478Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 71.387391ms iteration = 49 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6384	2024-12-06T21:22:54.479Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6385	2024-12-06T21:22:54.479Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 72.286095ms iteration = 17 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
6386	2024-12-06T21:22:54.479Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6387	2024-12-06T21:22:54.479Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6388	2024-12-06T21:22:54.481Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
6389	2024-12-06T21:22:54.481Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6390	2024-12-06T21:22:54.481Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6391	2024-12-06T21:22:54.481Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6392	2024-12-06T21:22:54.481Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6393	2024-12-06T21:22:54.481Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6394	2024-12-06T21:22:54.482Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6395	2024-12-06T21:22:54.482Z	DEBG	nexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
6396	2024-12-06T21:22:54.482Z	INFO	nexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
6397	2024-12-06T21:22:54.482Z	DEBG	nexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 75.462998ms iteration = 9 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
6398	2024-12-06T21:22:54.482Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
6399	2024-12-06T21:22:54.482Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
6400	2024-12-06T21:22:54.483Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
6401	2024-12-06T21:22:54.483Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
6402	2024-12-06T21:22:54.483Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6403	2024-12-06T21:22:54.484Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
6404	2024-12-06T21:22:54.484Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
6405	2024-12-06T21:22:54.484Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
6406	2024-12-06T21:22:54.484Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6407	2024-12-06T21:22:54.484Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
6408	2024-12-06T21:22:54.484Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6409	2024-12-06T21:22:54.484Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6410	2024-12-06T21:22:54.484Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6411	2024-12-06T21:22:54.485Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
6412	2024-12-06T21:22:54.485Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6413	2024-12-06T21:22:54.485Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6414	2024-12-06T21:22:54.485Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6415	2024-12-06T21:22:54.486Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6416	2024-12-06T21:22:54.488Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6417	2024-12-06T21:22:54.489Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6418	2024-12-06T21:22:54.489Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6419	2024-12-06T21:22:54.489Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6420	2024-12-06T21:22:54.489Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6421	2024-12-06T21:22:54.490Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6422	2024-12-06T21:22:54.493Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
6423	2024-12-06T21:22:54.493Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6424	2024-12-06T21:22:54.493Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6425	2024-12-06T21:22:54.494Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
6426	2024-12-06T21:22:54.494Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6427	2024-12-06T21:22:54.494Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6428	2024-12-06T21:22:54.502Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
6429	2024-12-06T21:22:54.502Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
6430	2024-12-06T21:22:54.505Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6431	2024-12-06T21:22:54.505Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6432	2024-12-06T21:22:54.505Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
6433	2024-12-06T21:22:54.505Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 98.262512ms iteration = 49 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
6434	2024-12-06T21:22:54.508Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 101.290101ms iteration = 17 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
6435	2024-12-06T21:22:54.509Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6436	2024-12-06T21:22:54.509Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6437	2024-12-06T21:22:54.511Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6438	2024-12-06T21:22:54.512Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6439	2024-12-06T21:22:54.513Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6440	2024-12-06T21:22:54.514Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
6441	2024-12-06T21:22:54.514Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
6442	2024-12-06T21:22:54.514Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
6443	2024-12-06T21:22:54.514Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6444	2024-12-06T21:22:54.514Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6445	2024-12-06T21:22:54.515Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6446	2024-12-06T21:22:54.516Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cbc5c1c6-5668-455a-85ef-94f4aebe0f80, lookup_type: ById(cbc5c1c6-5668-455a-85ef-94f4aebe0f80) } result = Ok(())
6447	2024-12-06T21:22:54.516Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6448	2024-12-06T21:22:54.516Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6449	2024-12-06T21:22:54.516Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6450	2024-12-06T21:22:54.519Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
6451	2024-12-06T21:22:54.519Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6452	2024-12-06T21:22:54.519Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
6453	2024-12-06T21:22:54.519Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6454	2024-12-06T21:22:54.519Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6455	2024-12-06T21:22:54.519Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6456	2024-12-06T21:22:54.522Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6457	2024-12-06T21:22:54.523Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 115.680088ms iteration = 17 status = 0
6458	2024-12-06T21:22:54.524Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
6459	2024-12-06T21:22:54.524Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6460	2024-12-06T21:22:54.524Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6461	2024-12-06T21:22:54.527Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6462	2024-12-06T21:22:54.527Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
6463	2024-12-06T21:22:54.527Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6464	2024-12-06T21:22:54.527Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6465	2024-12-06T21:22:54.527Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6466	2024-12-06T21:22:54.528Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
6467	2024-12-06T21:22:54.528Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 120.714291ms iteration = 17 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
6468	2024-12-06T21:22:54.530Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
6469	2024-12-06T21:22:54.530Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6470	2024-12-06T21:22:54.530Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6471	2024-12-06T21:22:54.530Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
6472	2024-12-06T21:22:54.530Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 93c3f865-bc00-42df-bef7-20e39342f84f dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
6473	2024-12-06T21:22:54.530Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6474	2024-12-06T21:22:54.530Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6475	2024-12-06T21:22:54.534Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 127.651043ms iteration = 18 status = {"physical_disks_added":0}
6476	2024-12-06T21:22:54.546Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 139.076941ms iteration = 17 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
6477	2024-12-06T21:22:54.551Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
6478	2024-12-06T21:22:54.557Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
6479	2024-12-06T21:22:54.557Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 05c53cbc-f66c-491f-af8d-6f291fe21596 dns_zone_name = oxide.test found_so_far = 0 version = 2
6480	2024-12-06T21:22:54.557Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6481	2024-12-06T21:22:54.558Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6482	2024-12-06T21:22:54.562Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6483	2024-12-06T21:22:54.562Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
6484	2024-12-06T21:22:54.562Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 154.82258ms iteration = 17 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
6485	2024-12-06T21:22:54.565Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6486	2024-12-06T21:22:54.568Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
6487	2024-12-06T21:22:54.568Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6488	2024-12-06T21:22:54.569Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6489	2024-12-06T21:22:54.570Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6490	2024-12-06T21:22:54.570Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6491	2024-12-06T21:22:54.571Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 05c53cbc-f66c-491f-af8d-6f291fe21596 dns_zone_name = oxide.test found_so_far = 1 version = 2
6492	2024-12-06T21:22:54.571Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
6493	2024-12-06T21:22:54.571Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-06 21:15:07.045426 UTC dns_group = external generation = 2
6494	2024-12-06T21:22:54.571Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 164.190284ms iteration = 9 status = {"generation":2}
6495	2024-12-06T21:22:54.572Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6496	2024-12-06T21:22:54.573Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6497	2024-12-06T21:22:54.574Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
6498	2024-12-06T21:22:54.574Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6499	2024-12-06T21:22:54.575Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6500	2024-12-06T21:22:54.577Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
6501	2024-12-06T21:22:54.577Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6502	2024-12-06T21:22:54.577Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6503	2024-12-06T21:22:54.580Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 93c3f865-bc00-42df-bef7-20e39342f84f dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
6504	2024-12-06T21:22:54.580Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
6505	2024-12-06T21:22:54.580Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-06 21:15:06.025861 UTC dns_group = internal generation = 1
6506	2024-12-06T21:22:54.580Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 173.285132ms iteration = 9 status = {"generation":1}
6507	2024-12-06T21:22:54.617Z	DEBG	nexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-06T21:14:50.566199Z, time_modified: 2024-12-06T21:14:50.566199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, db_silo: Silo { identity: SiloIdentity { id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-06T21:15:06.991365Z, time_modified: 2024-12-06T21:15:06.991365Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, db_silo: Silo { identity: SiloIdentity { id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-06T21:15:06.991365Z, time_modified: 2024-12-06T21:15:06.991365Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1" }] }) }
6508	2024-12-06T21:22:54.617Z	DEBG	nexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 209.73471ms iteration = 9 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"683d02e1-6ec8-48bf-98d0-db6c94785f80","tls_certs":["d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1"]}},"default_endpoint":{"silo_id":"683d02e1-6ec8-48bf-98d0-db6c94785f80","tls_certs":["d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
6509	2024-12-06T21:22:54.617Z	WARN	nexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
6510	2024-12-06T21:22:54.617Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 210.364541ms iteration = 9 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
6511	2024-12-06T21:22:54.697Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
6512	2024-12-06T21:22:54.697Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6513	2024-12-06T21:22:54.697Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa3582be-67e1-4770-a733-927f5e851863", "content-length": "26", "date": "Fri, 06 Dec 2024 21:22:54 GMT"} })
6514	2024-12-06T21:22:54.697Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6515	2024-12-06T21:22:54.698Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
6516	2024-12-06T21:22:54.698Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
6517	2024-12-06T21:22:54.727Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
6518	2024-12-06T21:22:54.727Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6519	2024-12-06T21:22:54.728Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "13492db1-6a2b-4f4b-96cb-2ceaa60c7469", "content-length": "26", "date": "Fri, 06 Dec 2024 21:22:54 GMT"} })
6520	2024-12-06T21:22:54.728Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6521	2024-12-06T21:22:54.729Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
6522	2024-12-06T21:22:54.746Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 zone_address = fd00:1122:3344:101::2
6523	2024-12-06T21:22:54.746Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6524	2024-12-06T21:22:54.746Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c695840-f21d-4a29-a7eb-8d1d1687e4db", "content-length": "26", "date": "Fri, 06 Dec 2024 21:22:54 GMT"} })
6525	2024-12-06T21:22:54.746Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6526	2024-12-06T21:22:54.747Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
6527	2024-12-06T21:22:54.766Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
6528	2024-12-06T21:22:54.766Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
6529	2024-12-06T21:22:54.766Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0cfd9f0-9de2-4a19-b2c8-8e02610619e5", "content-length": "372", "date": "Fri, 06 Dec 2024 21:22:53 GMT"} })
6530	2024-12-06T21:22:54.766Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6531	2024-12-06T21:22:54.766Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6532	2024-12-06T21:22:54.776Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
6533	2024-12-06T21:22:54.776Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "ae1d41c3-5882-4ed1-ba28-36ae49811e2a", "date": "Fri, 06 Dec 2024 21:22:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
6534	2024-12-06T21:22:54.776Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
6535	2024-12-06T21:22:54.778Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "da11638d-8899-4653-8c02-e8c02105f143", "date": "Fri, 06 Dec 2024 21:22:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
6536	2024-12-06T21:22:54.778Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
6537	2024-12-06T21:22:54.781Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
6538	2024-12-06T21:22:54.781Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 373.73594ms iteration = 11 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
6539	2024-12-06T21:22:54.803Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
6540	2024-12-06T21:22:54.804Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "582996a8-abd4-4ee3-a6c0-dbbc6dc0d715", "content-length": "691", "date": "Fri, 06 Dec 2024 21:22:53 GMT"} })
6541	2024-12-06T21:22:54.804Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
6542	2024-12-06T21:22:54.804Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
6543	2024-12-06T21:22:54.804Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
6544	2024-12-06T21:22:54.805Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 397.985848ms iteration = 17 status = {}
6545	2024-12-06T21:22:54.806Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
6546	2024-12-06T21:22:54.807Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
6547	2024-12-06T21:22:54.807Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "beee6fde-b713-4f14-8e10-244bb1d8550d", "date": "Fri, 06 Dec 2024 21:22:53 GMT"} })
6548	2024-12-06T21:22:54.807Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6549	2024-12-06T21:22:54.807Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6550	2024-12-06T21:22:54.809Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6551	2024-12-06T21:22:54.809Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6552	2024-12-06T21:22:54.812Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6553	2024-12-06T21:22:54.813Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6554	2024-12-06T21:22:54.816Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
6555	2024-12-06T21:22:54.816Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
6556	2024-12-06T21:22:54.816Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "64813a63-0a91-43f0-a4c2-4bf360dfac55", "date": "Fri, 06 Dec 2024 21:22:53 GMT"} })
6557	2024-12-06T21:22:54.818Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 410.758607ms iteration = 17 status = {}
6558	2024-12-06T21:22:54.872Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
6559	2024-12-06T21:22:54.873Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa02fd4c-9050-4f5a-8ee3-f20647e68d2a", "content-length": "1", "date": "Fri, 06 Dec 2024 21:22:53 GMT"} })
6560	2024-12-06T21:22:54.873Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 465.743739ms iteration = 17 status = {}
6561	2024-12-06T21:22:54.875Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
6562	2024-12-06T21:22:54.876Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6563	2024-12-06T21:22:54.877Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 470.162192ms iteration = 17 status = null
6564	2024-12-06T21:22:54.915Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
6565	2024-12-06T21:22:55.029Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
6566	2024-12-06T21:22:55.029Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "fea03c47-f87d-42a5-91b4-fed3b0e3b58b", "date": "Fri, 06 Dec 2024 21:22:55 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
6567	2024-12-06T21:22:55.029Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
6568	2024-12-06T21:22:55.031Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "28d65c80-9bdb-4903-bf30-ef2d817a6150", "date": "Fri, 06 Dec 2024 21:22:55 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
6569	2024-12-06T21:22:55.031Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
6570	2024-12-06T21:22:55.033Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "7259d293-6d20-4bd0-a771-202679c10849", "date": "Fri, 06 Dec 2024 21:22:55 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
6571	2024-12-06T21:22:55.033Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
6572	2024-12-06T21:22:55.035Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
6573	2024-12-06T21:22:55.035Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 628.063461ms iteration = 11 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
6574	2024-12-06T21:22:55.064Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6575	2024-12-06T21:22:55.064Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6576	2024-12-06T21:22:55.066Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 3d56b7bf-53ed-4147-b45b-9cef154d7338, rack_id: 2d75e4a3-6363-4fd1-935e-989d31fd7d71, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(02859c9b-92d7-4b19-b425-8f266763be54) } rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
6577	2024-12-06T21:22:55.066Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6578	2024-12-06T21:22:55.066Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6579	2024-12-06T21:22:55.075Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
6580	2024-12-06T21:22:55.076Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "e8d87100-e303-448f-9ec6-8cad11e8ecfe", "content-length": "134", "date": "Fri, 06 Dec 2024 21:22:54 GMT"} })
6581	2024-12-06T21:22:55.076Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
6582	2024-12-06T21:22:55.076Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 02859c9b-92d7-4b19-b425-8f266763be54
6583	2024-12-06T21:22:55.076Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
6584	2024-12-06T21:22:55.076Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
6585	2024-12-06T21:22:55.076Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
6586	2024-12-06T21:22:55.076Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
6587	2024-12-06T21:22:55.076Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
6588	2024-12-06T21:22:55.076Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
6589	2024-12-06T21:22:55.076Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
6590	2024-12-06T21:22:55.076Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
6591	2024-12-06T21:22:55.077Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
6592	2024-12-06T21:22:55.077Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6593	2024-12-06T21:22:55.077Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6594	2024-12-06T21:22:55.079Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
6595	2024-12-06T21:22:55.079Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "f43ae01a-a408-47f8-96c6-ca96941a611b", "content-length": "2", "date": "Fri, 06 Dec 2024 21:22:54 GMT"} })
6596	2024-12-06T21:22:55.079Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
6597	2024-12-06T21:22:55.079Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "fece6e4b-1fa4-4b12-9a0c-42718c0efcbd", "content-length": "52", "date": "Fri, 06 Dec 2024 21:22:55 GMT"} })
6598	2024-12-06T21:22:55.079Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
6599	2024-12-06T21:22:55.079Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
6600	2024-12-06T21:22:55.178Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "520b3046-dde7-4b4d-b698-3eee95956673", "date": "Fri, 06 Dec 2024 21:22:55 GMT"} })
6601	2024-12-06T21:22:55.178Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch = Switch0
6602	2024-12-06T21:22:55.178Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
6603	2024-12-06T21:22:55.178Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "3c9192f0-a49a-4493-a0ae-6c289acfc821", "content-length": "682", "date": "Fri, 06 Dec 2024 21:22:55 GMT"} })
6604	2024-12-06T21:22:55.178Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6605	2024-12-06T21:22:55.179Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6606	2024-12-06T21:22:55.182Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6607	2024-12-06T21:22:55.183Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6608	2024-12-06T21:22:55.186Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6609	2024-12-06T21:22:55.186Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6610	2024-12-06T21:22:55.188Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6611	2024-12-06T21:22:55.188Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6612	2024-12-06T21:22:55.195Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 787.499687ms iteration = 17 status = {}
6613	2024-12-06T21:22:56.541Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 302 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 856a6c12-d49e-4992-8c39-bfb205092eae response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
6614	2024-12-06T21:23:04.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 50 reason = Timeout
6615	2024-12-06T21:23:04.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 50 reason = Timeout
6616	2024-12-06T21:23:04.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6617	2024-12-06T21:23:04.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6618	2024-12-06T21:23:04.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6619	2024-12-06T21:23:04.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6620	2024-12-06T21:23:04.409Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6621	2024-12-06T21:23:04.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6622	2024-12-06T21:23:04.412Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6623	2024-12-06T21:23:04.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6624	2024-12-06T21:23:04.413Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6625	2024-12-06T21:23:04.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6626	2024-12-06T21:23:04.415Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6627	2024-12-06T21:23:04.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
6628	2024-12-06T21:23:04.418Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6629	2024-12-06T21:23:04.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6630	2024-12-06T21:23:04.425Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
6631	2024-12-06T21:23:04.425Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.712087ms iteration = 50 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
6632	2024-12-06T21:23:04.436Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6633	2024-12-06T21:23:04.437Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6634	2024-12-06T21:23:04.464Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 57.480612ms iteration = 50 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6635	2024-12-06T21:23:06.542Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 277 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 5d84abdd-3077-43ed-bdc5-875e21aec366 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
6636	2024-12-06T21:23:14.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 51 reason = Timeout
6637	2024-12-06T21:23:14.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 51 reason = Timeout
6638	2024-12-06T21:23:14.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6639	2024-12-06T21:23:14.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6640	2024-12-06T21:23:14.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6641	2024-12-06T21:23:14.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6642	2024-12-06T21:23:14.409Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6643	2024-12-06T21:23:14.410Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6644	2024-12-06T21:23:14.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6645	2024-12-06T21:23:14.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6646	2024-12-06T21:23:14.411Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6647	2024-12-06T21:23:14.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6648	2024-12-06T21:23:14.413Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6649	2024-12-06T21:23:14.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6650	2024-12-06T21:23:14.415Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6651	2024-12-06T21:23:14.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
6652	2024-12-06T21:23:14.417Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6653	2024-12-06T21:23:14.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6654	2024-12-06T21:23:14.428Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
6655	2024-12-06T21:23:14.428Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.747947ms iteration = 51 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
6656	2024-12-06T21:23:14.439Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.016205ms iteration = 51 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6657	2024-12-06T21:23:16.542Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 366 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = d5ee480b-d0b3-4b36-8c8a-ec0c265a3450 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
6658	2024-12-06T21:23:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 52 reason = Timeout
6659	2024-12-06T21:23:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 18 reason = Timeout
6660	2024-12-06T21:23:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6661	2024-12-06T21:23:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 18 reason = Timeout
6662	2024-12-06T21:23:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6663	2024-12-06T21:23:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
6664	2024-12-06T21:23:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 19 reason = Timeout
6665	2024-12-06T21:23:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 18 reason = Timeout
6666	2024-12-06T21:23:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
6667	2024-12-06T21:23:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6668	2024-12-06T21:23:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 18 reason = Timeout
6669	2024-12-06T21:23:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
6670	2024-12-06T21:23:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 18 reason = Timeout
6671	2024-12-06T21:23:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
6672	2024-12-06T21:23:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 18 reason = Timeout
6673	2024-12-06T21:23:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 18 reason = Timeout
6674	2024-12-06T21:23:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
6675	2024-12-06T21:23:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6676	2024-12-06T21:23:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 18 reason = Timeout
6677	2024-12-06T21:23:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
6678	2024-12-06T21:23:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 18 reason = Timeout
6679	2024-12-06T21:23:24.407Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
6680	2024-12-06T21:23:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6681	2024-12-06T21:23:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 18 reason = Timeout
6682	2024-12-06T21:23:24.407Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
6683	2024-12-06T21:23:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6684	2024-12-06T21:23:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 18 reason = Timeout
6685	2024-12-06T21:23:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6686	2024-12-06T21:23:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 18 reason = Timeout
6687	2024-12-06T21:23:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 52 reason = Timeout
6688	2024-12-06T21:23:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6689	2024-12-06T21:23:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6690	2024-12-06T21:23:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 18 reason = Timeout
6691	2024-12-06T21:23:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 18 reason = Timeout
6692	2024-12-06T21:23:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
6693	2024-12-06T21:23:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6694	2024-12-06T21:23:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
6695	2024-12-06T21:23:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6696	2024-12-06T21:23:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
6697	2024-12-06T21:23:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6698	2024-12-06T21:23:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6699	2024-12-06T21:23:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6700	2024-12-06T21:23:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
6701	2024-12-06T21:23:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
6702	2024-12-06T21:23:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6703	2024-12-06T21:23:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
6704	2024-12-06T21:23:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
6705	2024-12-06T21:23:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6706	2024-12-06T21:23:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6707	2024-12-06T21:23:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6708	2024-12-06T21:23:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6709	2024-12-06T21:23:24.410Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6710	2024-12-06T21:23:24.410Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1319, tv_nsec: 908362500 } })
6711	2024-12-06T21:23:24.410Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.01232ms iteration = 18 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
6712	2024-12-06T21:23:24.410Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
6713	2024-12-06T21:23:24.411Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6714	2024-12-06T21:23:24.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6715	2024-12-06T21:23:24.413Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6716	2024-12-06T21:23:24.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
6717	2024-12-06T21:23:24.413Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
6718	2024-12-06T21:23:24.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6719	2024-12-06T21:23:24.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
6720	2024-12-06T21:23:24.416Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6721	2024-12-06T21:23:24.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6722	2024-12-06T21:23:24.418Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6723	2024-12-06T21:23:24.420Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
6724	2024-12-06T21:23:24.420Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6725	2024-12-06T21:23:24.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6726	2024-12-06T21:23:24.426Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1319, tv_nsec: 924737458 } })
6727	2024-12-06T21:23:24.427Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
6728	2024-12-06T21:23:24.429Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6729	2024-12-06T21:23:24.429Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6730	2024-12-06T21:23:24.435Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6731	2024-12-06T21:23:24.436Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6732	2024-12-06T21:23:24.436Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6733	2024-12-06T21:23:24.440Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
6734	2024-12-06T21:23:24.440Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6735	2024-12-06T21:23:24.440Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6736	2024-12-06T21:23:24.445Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 37.981893ms iteration = 18 status = {"errors":[],"garbage_collect_requested":[]}
6737	2024-12-06T21:23:24.447Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6738	2024-12-06T21:23:24.448Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6739	2024-12-06T21:23:24.449Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
6740	2024-12-06T21:23:24.449Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6741	2024-12-06T21:23:24.449Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6742	2024-12-06T21:23:24.450Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
6743	2024-12-06T21:23:24.450Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6744	2024-12-06T21:23:24.450Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "31adff33-9fba-4be7-9ee7-01595704c453", "content-length": "26", "date": "Fri, 06 Dec 2024 21:23:24 GMT"} })
6745	2024-12-06T21:23:24.450Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6746	2024-12-06T21:23:24.451Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6747	2024-12-06T21:23:24.452Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
6748	2024-12-06T21:23:24.452Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6749	2024-12-06T21:23:24.452Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6750	2024-12-06T21:23:24.452Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
6751	2024-12-06T21:23:24.452Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6752	2024-12-06T21:23:24.453Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6753	2024-12-06T21:23:24.453Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
6754	2024-12-06T21:23:24.453Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
6755	2024-12-06T21:23:24.454Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
6756	2024-12-06T21:23:24.454Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 46.677416ms iteration = 52 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
6757	2024-12-06T21:23:24.454Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
6758	2024-12-06T21:23:24.454Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6759	2024-12-06T21:23:24.454Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6760	2024-12-06T21:23:24.455Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
6761	2024-12-06T21:23:24.455Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
6762	2024-12-06T21:23:24.456Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 49.3854ms iteration = 18 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
6763	2024-12-06T21:23:24.457Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6764	2024-12-06T21:23:24.457Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
6765	2024-12-06T21:23:24.457Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6766	2024-12-06T21:23:24.457Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6767	2024-12-06T21:23:24.459Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6768	2024-12-06T21:23:24.459Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 52.4339ms iteration = 18 status = {"errors":[],"records_set_to_done":[]}
6769	2024-12-06T21:23:24.459Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6770	2024-12-06T21:23:24.460Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6771	2024-12-06T21:23:24.460Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
6772	2024-12-06T21:23:24.460Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6773	2024-12-06T21:23:24.460Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6774	2024-12-06T21:23:24.462Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6775	2024-12-06T21:23:24.463Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6776	2024-12-06T21:23:24.463Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 56.197912ms iteration = 52 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6777	2024-12-06T21:23:24.463Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6778	2024-12-06T21:23:24.465Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6779	2024-12-06T21:23:24.465Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6780	2024-12-06T21:23:24.465Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
6781	2024-12-06T21:23:24.465Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 58.080502ms iteration = 18 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
6782	2024-12-06T21:23:24.465Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
6783	2024-12-06T21:23:24.465Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6784	2024-12-06T21:23:24.465Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6785	2024-12-06T21:23:24.467Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6786	2024-12-06T21:23:24.467Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
6787	2024-12-06T21:23:24.467Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6788	2024-12-06T21:23:24.467Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6b15338e-d162-4ea3-8757-1896808772f2", "content-length": "26", "date": "Fri, 06 Dec 2024 21:23:24 GMT"} })
6789	2024-12-06T21:23:24.467Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6790	2024-12-06T21:23:24.468Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1319, tv_nsec: 966443642 } })
6791	2024-12-06T21:23:24.468Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
6792	2024-12-06T21:23:24.468Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
6793	2024-12-06T21:23:24.469Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cbc5c1c6-5668-455a-85ef-94f4aebe0f80, lookup_type: ById(cbc5c1c6-5668-455a-85ef-94f4aebe0f80) } result = Ok(())
6794	2024-12-06T21:23:24.469Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6795	2024-12-06T21:23:24.470Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6796	2024-12-06T21:23:24.476Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 68.782718ms iteration = 18 status = 0
6797	2024-12-06T21:23:24.494Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6798	2024-12-06T21:23:24.496Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
6799	2024-12-06T21:23:24.496Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6800	2024-12-06T21:23:24.496Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6801	2024-12-06T21:23:24.497Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 89.924026ms iteration = 18 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
6802	2024-12-06T21:23:24.504Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6803	2024-12-06T21:23:24.504Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6804	2024-12-06T21:23:24.506Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6805	2024-12-06T21:23:24.506Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
6806	2024-12-06T21:23:24.506Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 98.559266ms iteration = 18 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
6807	2024-12-06T21:23:24.527Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 119.781945ms iteration = 19 status = {"physical_disks_added":0}
6808	2024-12-06T21:23:24.531Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6809	2024-12-06T21:23:24.531Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6810	2024-12-06T21:23:24.532Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 124.667875ms iteration = 18 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
6811	2024-12-06T21:23:24.540Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 132.631715ms iteration = 18 status = {}
6812	2024-12-06T21:23:24.572Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6813	2024-12-06T21:23:24.573Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6814	2024-12-06T21:23:24.574Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6815	2024-12-06T21:23:24.576Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
6816	2024-12-06T21:23:24.576Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6817	2024-12-06T21:23:24.576Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6818	2024-12-06T21:23:24.616Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
6819	2024-12-06T21:23:24.616Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
6820	2024-12-06T21:23:24.617Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a0c8f7c-cc40-4881-9e26-285a59514053", "content-length": "372", "date": "Fri, 06 Dec 2024 21:23:23 GMT"} })
6821	2024-12-06T21:23:24.617Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6822	2024-12-06T21:23:24.617Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6823	2024-12-06T21:23:24.618Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6824	2024-12-06T21:23:24.621Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
6825	2024-12-06T21:23:24.621Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6826	2024-12-06T21:23:24.621Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6827	2024-12-06T21:23:24.625Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 zone_address = fd00:1122:3344:101::2
6828	2024-12-06T21:23:24.625Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6829	2024-12-06T21:23:24.625Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1ef660b9-3bde-4761-b762-3257c6495d37", "content-length": "26", "date": "Fri, 06 Dec 2024 21:23:24 GMT"} })
6830	2024-12-06T21:23:24.625Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6831	2024-12-06T21:23:24.627Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
6832	2024-12-06T21:23:24.634Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
6833	2024-12-06T21:23:24.634Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
6834	2024-12-06T21:23:24.634Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "996040cf-97d3-4f4a-b945-1f3e7573a87d", "date": "Fri, 06 Dec 2024 21:23:23 GMT"} })
6835	2024-12-06T21:23:24.634Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6836	2024-12-06T21:23:24.634Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6837	2024-12-06T21:23:24.637Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6838	2024-12-06T21:23:24.637Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6839	2024-12-06T21:23:24.640Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6840	2024-12-06T21:23:24.640Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6841	2024-12-06T21:23:24.644Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
6842	2024-12-06T21:23:24.644Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
6843	2024-12-06T21:23:24.644Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "761ccff3-91fb-4377-8f8c-6842b7130af8", "date": "Fri, 06 Dec 2024 21:23:23 GMT"} })
6844	2024-12-06T21:23:24.646Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 238.848358ms iteration = 18 status = {}
6845	2024-12-06T21:23:24.671Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
6846	2024-12-06T21:23:24.671Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "cff6ef96-ecfe-4892-bdaf-2f6aea0d1ab3", "content-length": "1", "date": "Fri, 06 Dec 2024 21:23:24 GMT"} })
6847	2024-12-06T21:23:24.673Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
6848	2024-12-06T21:23:24.673Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6849	2024-12-06T21:23:24.723Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
6850	2024-12-06T21:23:24.724Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "4969703f-ceb3-43e5-91d8-ed796d81f815", "content-length": "691", "date": "Fri, 06 Dec 2024 21:23:23 GMT"} })
6851	2024-12-06T21:23:24.724Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
6852	2024-12-06T21:23:24.724Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
6853	2024-12-06T21:23:24.724Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
6854	2024-12-06T21:23:24.726Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 318.397652ms iteration = 18 status = null
6855	2024-12-06T21:23:24.726Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 318.818009ms iteration = 18 status = {}
6856	2024-12-06T21:23:24.798Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6857	2024-12-06T21:23:24.799Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6858	2024-12-06T21:23:24.801Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 3d56b7bf-53ed-4147-b45b-9cef154d7338, rack_id: 2d75e4a3-6363-4fd1-935e-989d31fd7d71, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(02859c9b-92d7-4b19-b425-8f266763be54) } rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
6859	2024-12-06T21:23:24.801Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6860	2024-12-06T21:23:24.801Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6861	2024-12-06T21:23:24.815Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
6862	2024-12-06T21:23:24.815Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "0c74c041-31b2-4737-994a-b7ce406ff4d5", "content-length": "134", "date": "Fri, 06 Dec 2024 21:23:24 GMT"} })
6863	2024-12-06T21:23:24.815Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
6864	2024-12-06T21:23:24.815Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 02859c9b-92d7-4b19-b425-8f266763be54
6865	2024-12-06T21:23:24.816Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
6866	2024-12-06T21:23:24.816Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
6867	2024-12-06T21:23:24.816Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
6868	2024-12-06T21:23:24.816Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
6869	2024-12-06T21:23:24.816Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
6870	2024-12-06T21:23:24.816Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
6871	2024-12-06T21:23:24.816Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
6872	2024-12-06T21:23:24.816Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
6873	2024-12-06T21:23:24.816Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
6874	2024-12-06T21:23:24.816Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6875	2024-12-06T21:23:24.817Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6876	2024-12-06T21:23:24.819Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
6877	2024-12-06T21:23:24.819Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba642643-8cab-4564-a794-70d3b7449d7b", "content-length": "2", "date": "Fri, 06 Dec 2024 21:23:24 GMT"} })
6878	2024-12-06T21:23:24.819Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
6879	2024-12-06T21:23:24.819Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "03e19b76-9faf-4936-bb5e-e96b8ed32828", "content-length": "52", "date": "Fri, 06 Dec 2024 21:23:24 GMT"} })
6880	2024-12-06T21:23:24.820Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
6881	2024-12-06T21:23:24.820Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
6882	2024-12-06T21:23:24.945Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "1cfa213d-c4f7-47f5-9c0a-ca38b3fa48df", "date": "Fri, 06 Dec 2024 21:23:24 GMT"} })
6883	2024-12-06T21:23:24.945Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch = Switch0
6884	2024-12-06T21:23:24.945Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
6885	2024-12-06T21:23:24.945Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "b55a1a75-907b-4fbf-9067-0fb10b3d6efa", "content-length": "682", "date": "Fri, 06 Dec 2024 21:23:24 GMT"} })
6886	2024-12-06T21:23:24.945Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6887	2024-12-06T21:23:24.946Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6888	2024-12-06T21:23:24.949Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6889	2024-12-06T21:23:24.949Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6890	2024-12-06T21:23:24.953Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6891	2024-12-06T21:23:24.953Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6892	2024-12-06T21:23:24.956Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6893	2024-12-06T21:23:24.956Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6894	2024-12-06T21:23:24.963Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 556.099901ms iteration = 18 status = {}
6895	2024-12-06T21:23:25.055Z	DEBG	nexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
6896	2024-12-06T21:23:25.055Z	DEBG	nexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
6897	2024-12-06T21:23:25.085Z	DEBG	nexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
6898	2024-12-06T21:23:25.085Z	INFO	nexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:58115
6899	2024-12-06T21:23:25.085Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:58115 req_id = cdf0201d-1f1a-4467-96b2-dd9778511fbe roles = RoleSet { roles: {} } uri = /metrics/producers
6900	2024-12-06T21:23:25.086Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:58115 req_id = cdf0201d-1f1a-4467-96b2-dd9778511fbe resource = Database result = Ok(()) uri = /metrics/producers
6901	2024-12-06T21:23:25.124Z	INFO	nexus: assigned collector to new producer collector_id = 9d509eff-9671-46fb-8274-6ec9019a9d9e file = nexus/src/app/oximeter.rs:76 producer_id = e8725c8c-13ab-4916-bf16-bb7f832014d5
6902	2024-12-06T21:23:25.124Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 39073 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:58115 req_id = cdf0201d-1f1a-4467-96b2-dd9778511fbe response_code = 201 uri = /metrics/producers
6903	2024-12-06T21:23:25.125Z	DEBG	nexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "cdf0201d-1f1a-4467-96b2-dd9778511fbe", "content-length": "41", "date": "Fri, 06 Dec 2024 21:23:25 GMT"} })
6904	2024-12-06T21:23:25.127Z	DEBG	nexus (producer-registration-task): registered with nexus successfully
6905	2024-12-06T21:23:25.127Z	DEBG	nexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
6906	2024-12-06T21:23:26.541Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 321 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 43942951-3a76-4a67-a704-bdfbaed4ca53 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
6907	2024-12-06T21:23:34.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 53 reason = Timeout
6908	2024-12-06T21:23:34.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 53 reason = Timeout
6909	2024-12-06T21:23:34.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6910	2024-12-06T21:23:34.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6911	2024-12-06T21:23:34.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6912	2024-12-06T21:23:34.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6913	2024-12-06T21:23:34.409Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6914	2024-12-06T21:23:34.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6915	2024-12-06T21:23:34.411Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6916	2024-12-06T21:23:34.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6917	2024-12-06T21:23:34.414Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6918	2024-12-06T21:23:34.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6919	2024-12-06T21:23:34.415Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6920	2024-12-06T21:23:34.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
6921	2024-12-06T21:23:34.418Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6922	2024-12-06T21:23:34.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6923	2024-12-06T21:23:34.429Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
6924	2024-12-06T21:23:34.429Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.627722ms iteration = 53 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
6925	2024-12-06T21:23:34.436Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6926	2024-12-06T21:23:34.436Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6927	2024-12-06T21:23:34.471Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 63.759796ms iteration = 53 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6928	2024-12-06T21:23:36.541Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 295 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 79698339-eff9-446b-9d4c-e46815756cfc response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
6929	2024-12-06T21:23:44.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 54 reason = Timeout
6930	2024-12-06T21:23:44.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 54 reason = Timeout
6931	2024-12-06T21:23:44.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6932	2024-12-06T21:23:44.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6933	2024-12-06T21:23:44.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6934	2024-12-06T21:23:44.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6935	2024-12-06T21:23:44.409Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6936	2024-12-06T21:23:44.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6937	2024-12-06T21:23:44.411Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6938	2024-12-06T21:23:44.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6939	2024-12-06T21:23:44.413Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6940	2024-12-06T21:23:44.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6941	2024-12-06T21:23:44.435Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6942	2024-12-06T21:23:44.435Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6943	2024-12-06T21:23:44.437Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 30.063413ms iteration = 54 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6944	2024-12-06T21:23:44.442Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6945	2024-12-06T21:23:44.445Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
6946	2024-12-06T21:23:44.445Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6947	2024-12-06T21:23:44.445Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6948	2024-12-06T21:23:44.456Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
6949	2024-12-06T21:23:44.456Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 49.736986ms iteration = 54 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
6950	2024-12-06T21:23:46.541Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 437 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = a722a3aa-acdf-4cdd-ac26-5acee835a7a1 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
6951	2024-12-06T21:23:51.535Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54925 req_id = fd181601-9bb3-4e73-a0a6-1b84349e7c84 roles = RoleSet { roles: {} } uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
6952	2024-12-06T21:23:51.536Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54925 req_id = fd181601-9bb3-4e73-a0a6-1b84349e7c84 resource = Database result = Ok(()) uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
6953	2024-12-06T21:23:51.539Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3408 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54925 req_id = fd181601-9bb3-4e73-a0a6-1b84349e7c84 response_code = 200 uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
6954	2024-12-06T21:23:51.539Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54925 req_id = b61baa99-62ee-4182-8c9a-1844ae332a78 roles = RoleSet { roles: {} } uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
6955	2024-12-06T21:23:51.539Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54925 req_id = b61baa99-62ee-4182-8c9a-1844ae332a78 resource = Database result = Ok(()) uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
6956	2024-12-06T21:23:51.542Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2640 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54925 req_id = b61baa99-62ee-4182-8c9a-1844ae332a78 response_code = 200 uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
6957	2024-12-06T21:23:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 12 reason = Timeout
6958	2024-12-06T21:23:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 12 reason = Timeout
6959	2024-12-06T21:23:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 19 reason = Timeout
6960	2024-12-06T21:23:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 55 reason = Timeout
6961	2024-12-06T21:23:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6962	2024-12-06T21:23:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 19 reason = Timeout
6963	2024-12-06T21:23:54.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6964	2024-12-06T21:23:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 11 reason = Timeout
6965	2024-12-06T21:23:54.406Z	WARN	nexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0
6966	2024-12-06T21:23:54.406Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 13.17\u{b5}s iteration = 11 status = {"enabled":false,"target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0"}
6967	2024-12-06T21:23:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 10 reason = Timeout
6968	2024-12-06T21:23:54.406Z	INFO	nexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
6969	2024-12-06T21:23:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 19 reason = Timeout
6970	2024-12-06T21:23:54.406Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
6971	2024-12-06T21:23:54.406Z	DEBG	nexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 10 reason = Timeout
6972	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
6973	2024-12-06T21:23:54.407Z	INFO	nexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-06 21:13:54.406931158 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
6974	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6975	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
6976	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 19 reason = Timeout
6977	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 19 reason = Timeout
6978	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
6979	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
6980	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 10 reason = Timeout
6981	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
6982	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 19 reason = Timeout
6983	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 19 reason = Timeout
6984	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6985	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6986	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 19 reason = Timeout
6987	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
6988	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = external_endpoints iteration = 10 reason = Timeout
6989	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6990	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 10 reason = Timeout
6991	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
6992	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 5 reason = Timeout
6993	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 10 reason = Timeout
6994	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
6995	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
6996	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 19 reason = Timeout
6997	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
6998	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 10 reason = Timeout
6999	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
7000	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7001	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 19 reason = Timeout
7002	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
7003	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 10 reason = Timeout
7004	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
7005	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7006	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 20 reason = Timeout
7007	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 19 reason = Timeout
7008	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7009	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
7010	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 55 reason = Timeout
7011	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7012	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 10 reason = Timeout
7013	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 19 reason = Timeout
7014	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
7015	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 19 reason = Timeout
7016	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
7017	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 19 reason = Timeout
7018	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
7019	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
7020	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 10 reason = Timeout
7021	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7022	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7023	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
7024	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7025	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7026	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7027	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7028	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
7029	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
7030	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
7031	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
7032	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
7033	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
7034	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
7035	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
7036	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7037	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
7038	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7039	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
7040	2024-12-06T21:23:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7041	2024-12-06T21:23:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7042	2024-12-06T21:23:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7043	2024-12-06T21:23:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
7044	2024-12-06T21:23:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
7045	2024-12-06T21:23:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
7046	2024-12-06T21:23:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
7047	2024-12-06T21:23:54.408Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4aff1ae3-2d71-40bf-bc46-79a7276da3c2.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4dccdb26-9cf9-40aa-9229-5d35905cf0e1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("79d475dc-44ec-44f4-9e51-ea8a753c39f3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("79d475dc-44ec-44f4-9e51-ea8a753c39f3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1349, tv_nsec: 905763223 } })
7048	2024-12-06T21:23:54.408Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("709da000-e0da-4b29-8160-3112fd080927.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a03854a6-7a78-402b-93b4-65c6d36bf236.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a03854a6-7a78-402b-93b4-65c6d36bf236.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1349, tv_nsec: 905769793 } })
7049	2024-12-06T21:23:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
7050	2024-12-06T21:23:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
7051	2024-12-06T21:23:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
7052	2024-12-06T21:23:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
7053	2024-12-06T21:23:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
7054	2024-12-06T21:23:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
7055	2024-12-06T21:23:54.408Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
7056	2024-12-06T21:23:54.408Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.23274ms iteration = 10 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
7057	2024-12-06T21:23:54.408Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
7058	2024-12-06T21:23:54.408Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.23218ms iteration = 10 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
7059	2024-12-06T21:23:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
7060	2024-12-06T21:23:54.409Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7061	2024-12-06T21:23:54.410Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7062	2024-12-06T21:23:54.411Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7063	2024-12-06T21:23:54.411Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.460023ms iteration = 19 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
7064	2024-12-06T21:23:54.411Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7065	2024-12-06T21:23:54.411Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7066	2024-12-06T21:23:54.412Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
7067	2024-12-06T21:23:54.412Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7068	2024-12-06T21:23:54.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7069	2024-12-06T21:23:54.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
7070	2024-12-06T21:23:54.412Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7071	2024-12-06T21:23:54.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7072	2024-12-06T21:23:54.414Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1349, tv_nsec: 911914124 } })
7073	2024-12-06T21:23:54.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
7074	2024-12-06T21:23:54.414Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
7075	2024-12-06T21:23:54.414Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
7076	2024-12-06T21:23:54.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
7077	2024-12-06T21:23:54.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7078	2024-12-06T21:23:54.415Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7079	2024-12-06T21:23:54.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
7080	2024-12-06T21:23:54.415Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7081	2024-12-06T21:23:54.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7082	2024-12-06T21:23:54.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7083	2024-12-06T21:23:54.416Z	DEBG	nexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 9.392524ms iteration = 10 status = {"errors":[],"found_port_ok":[]}
7084	2024-12-06T21:23:54.417Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7085	2024-12-06T21:23:54.418Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1349, tv_nsec: 916231455 } })
7086	2024-12-06T21:23:54.418Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
7087	2024-12-06T21:23:54.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
7088	2024-12-06T21:23:54.419Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
7089	2024-12-06T21:23:54.419Z	INFO	nexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-06 21:13:54.406931158 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
7090	2024-12-06T21:23:54.419Z	DEBG	nexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 12.217071ms iteration = 10 status = {"expiration":"2024-12-06T21:13:54.406931158Z","pruned":[]}
7091	2024-12-06T21:23:54.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
7092	2024-12-06T21:23:54.421Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7093	2024-12-06T21:23:54.421Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7094	2024-12-06T21:23:54.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7095	2024-12-06T21:23:54.423Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
7096	2024-12-06T21:23:54.423Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
7097	2024-12-06T21:23:54.423Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
7098	2024-12-06T21:23:54.423Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7099	2024-12-06T21:23:54.424Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7100	2024-12-06T21:23:54.434Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7101	2024-12-06T21:23:54.436Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
7102	2024-12-06T21:23:54.436Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
7103	2024-12-06T21:23:54.436Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
7104	2024-12-06T21:23:54.439Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1349, tv_nsec: 937086867 } })
7105	2024-12-06T21:23:54.439Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
7106	2024-12-06T21:23:54.441Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7107	2024-12-06T21:23:54.441Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7108	2024-12-06T21:23:54.462Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7109	2024-12-06T21:23:54.465Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
7110	2024-12-06T21:23:54.465Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7111	2024-12-06T21:23:54.465Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7112	2024-12-06T21:23:54.466Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
7113	2024-12-06T21:23:54.466Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
7114	2024-12-06T21:23:54.466Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7115	2024-12-06T21:23:54.467Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
7116	2024-12-06T21:23:54.468Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
7117	2024-12-06T21:23:54.469Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
7118	2024-12-06T21:23:54.469Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
7119	2024-12-06T21:23:54.469Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
7120	2024-12-06T21:23:54.470Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7121	2024-12-06T21:23:54.471Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7122	2024-12-06T21:23:54.472Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7123	2024-12-06T21:23:54.474Z	DEBG	nexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
7124	2024-12-06T21:23:54.474Z	INFO	nexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
7125	2024-12-06T21:23:54.474Z	DEBG	nexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 67.527938ms iteration = 10 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
7126	2024-12-06T21:23:54.474Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7127	2024-12-06T21:23:54.474Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
7128	2024-12-06T21:23:54.475Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
7129	2024-12-06T21:23:54.476Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
7130	2024-12-06T21:23:54.476Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7131	2024-12-06T21:23:54.478Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
7132	2024-12-06T21:23:54.478Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7133	2024-12-06T21:23:54.478Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
7134	2024-12-06T21:23:54.478Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7135	2024-12-06T21:23:54.478Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7136	2024-12-06T21:23:54.478Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7137	2024-12-06T21:23:54.480Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7138	2024-12-06T21:23:54.481Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7139	2024-12-06T21:23:54.481Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7140	2024-12-06T21:23:54.482Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
7141	2024-12-06T21:23:54.482Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7142	2024-12-06T21:23:54.482Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7143	2024-12-06T21:23:54.488Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7144	2024-12-06T21:23:54.488Z	DEBG	nexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
7145	2024-12-06T21:23:54.488Z	INFO	nexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
7146	2024-12-06T21:23:54.488Z	DEBG	nexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 81.591878ms iteration = 10 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
7147	2024-12-06T21:23:54.488Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7148	2024-12-06T21:23:54.489Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
7149	2024-12-06T21:23:54.489Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
7150	2024-12-06T21:23:54.489Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
7151	2024-12-06T21:23:54.489Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 82.35641ms iteration = 19 status = {"errors":[],"garbage_collect_requested":[]}
7152	2024-12-06T21:23:54.489Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
7153	2024-12-06T21:23:54.491Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 84.030749ms iteration = 19 status = {"errors":[],"records_set_to_done":[]}
7154	2024-12-06T21:23:54.494Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7155	2024-12-06T21:23:54.495Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7156	2024-12-06T21:23:54.495Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7157	2024-12-06T21:23:54.498Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
7158	2024-12-06T21:23:54.498Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
7159	2024-12-06T21:23:54.498Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
7160	2024-12-06T21:23:54.498Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7161	2024-12-06T21:23:54.499Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
7162	2024-12-06T21:23:54.499Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
7163	2024-12-06T21:23:54.499Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
7164	2024-12-06T21:23:54.499Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
7165	2024-12-06T21:23:54.500Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
7166	2024-12-06T21:23:54.500Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
7167	2024-12-06T21:23:54.500Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
7168	2024-12-06T21:23:54.505Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
7169	2024-12-06T21:23:54.505Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 98.287402ms iteration = 55 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
7170	2024-12-06T21:23:54.506Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
7171	2024-12-06T21:23:54.507Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
7172	2024-12-06T21:23:54.513Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7173	2024-12-06T21:23:54.516Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
7174	2024-12-06T21:23:54.517Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7175	2024-12-06T21:23:54.518Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7176	2024-12-06T21:23:54.519Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 112.82014ms iteration = 19 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
7177	2024-12-06T21:23:54.520Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
7178	2024-12-06T21:23:54.520Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 112.995093ms iteration = 19 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
7179	2024-12-06T21:23:54.520Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 113.087495ms iteration = 19 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
7180	2024-12-06T21:23:54.526Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7181	2024-12-06T21:23:54.529Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
7182	2024-12-06T21:23:54.529Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
7183	2024-12-06T21:23:54.529Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
7184	2024-12-06T21:23:54.529Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7185	2024-12-06T21:23:54.529Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7186	2024-12-06T21:23:54.530Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
7187	2024-12-06T21:23:54.530Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7188	2024-12-06T21:23:54.531Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7189	2024-12-06T21:23:54.532Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
7190	2024-12-06T21:23:54.532Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 124.998091ms iteration = 55 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
7191	2024-12-06T21:23:54.532Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
7192	2024-12-06T21:23:54.546Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
7193	2024-12-06T21:23:54.546Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
7194	2024-12-06T21:23:54.546Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
7195	2024-12-06T21:23:54.546Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7196	2024-12-06T21:23:54.547Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7197	2024-12-06T21:23:54.550Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7198	2024-12-06T21:23:54.550Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7199	2024-12-06T21:23:54.557Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
7200	2024-12-06T21:23:54.600Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 zone_address = fd00:1122:3344:101::2
7201	2024-12-06T21:23:54.600Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7202	2024-12-06T21:23:54.600Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba07f94a-08dc-4697-8382-a5ee1ac18156", "content-length": "26", "date": "Fri, 06 Dec 2024 21:23:54 GMT"} })
7203	2024-12-06T21:23:54.600Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7204	2024-12-06T21:23:54.602Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
7205	2024-12-06T21:23:54.729Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
7206	2024-12-06T21:23:54.729Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
7207	2024-12-06T21:23:54.729Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc1c347d-ef75-4648-8d2e-4d3c834d4d29", "content-length": "372", "date": "Fri, 06 Dec 2024 21:23:53 GMT"} })
7208	2024-12-06T21:23:54.729Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7209	2024-12-06T21:23:54.730Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7210	2024-12-06T21:23:54.731Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7211	2024-12-06T21:23:54.733Z	DEBG	nexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 326.18465ms iteration = 5 status = {"nsuccess":5}
7212	2024-12-06T21:23:54.733Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
7213	2024-12-06T21:23:54.733Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7214	2024-12-06T21:23:54.734Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7215	2024-12-06T21:23:54.736Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
7216	2024-12-06T21:23:54.736Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7217	2024-12-06T21:23:54.736Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7218	2024-12-06T21:23:54.736Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7219	2024-12-06T21:23:54.741Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
7220	2024-12-06T21:23:54.741Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7221	2024-12-06T21:23:54.741Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7222	2024-12-06T21:23:54.779Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
7223	2024-12-06T21:23:54.779Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
7224	2024-12-06T21:23:54.779Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 372.477359ms iteration = 19 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
7225	2024-12-06T21:23:54.782Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7226	2024-12-06T21:23:54.786Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7227	2024-12-06T21:23:54.786Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7228	2024-12-06T21:23:54.787Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
7229	2024-12-06T21:23:54.787Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7230	2024-12-06T21:23:54.787Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7231	2024-12-06T21:23:54.789Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 382.039696ms iteration = 19 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
7232	2024-12-06T21:23:54.789Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
7233	2024-12-06T21:23:54.789Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 93c3f865-bc00-42df-bef7-20e39342f84f dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
7234	2024-12-06T21:23:54.789Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7235	2024-12-06T21:23:54.790Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7236	2024-12-06T21:23:54.791Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7237	2024-12-06T21:23:54.795Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
7238	2024-12-06T21:23:54.795Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7239	2024-12-06T21:23:54.795Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7240	2024-12-06T21:23:54.795Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
7241	2024-12-06T21:23:54.797Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
7242	2024-12-06T21:23:54.797Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 05c53cbc-f66c-491f-af8d-6f291fe21596 dns_zone_name = oxide.test found_so_far = 0 version = 2
7243	2024-12-06T21:23:54.797Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7244	2024-12-06T21:23:54.797Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 93c3f865-bc00-42df-bef7-20e39342f84f dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
7245	2024-12-06T21:23:54.797Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
7246	2024-12-06T21:23:54.797Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-06 21:15:06.025861 UTC dns_group = internal generation = 1
7247	2024-12-06T21:23:54.797Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 390.541486ms iteration = 10 status = {"generation":1}
7248	2024-12-06T21:23:54.797Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7249	2024-12-06T21:23:54.804Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7250	2024-12-06T21:23:54.807Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cbc5c1c6-5668-455a-85ef-94f4aebe0f80, lookup_type: ById(cbc5c1c6-5668-455a-85ef-94f4aebe0f80) } result = Ok(())
7251	2024-12-06T21:23:54.807Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7252	2024-12-06T21:23:54.807Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7253	2024-12-06T21:23:54.810Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 403.396396ms iteration = 19 status = 0
7254	2024-12-06T21:23:54.811Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
7255	2024-12-06T21:23:54.811Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
7256	2024-12-06T21:23:54.812Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "2f6e45a1-d3c5-43fa-ab27-af3bb7ef885b", "date": "Fri, 06 Dec 2024 21:23:53 GMT"} })
7257	2024-12-06T21:23:54.812Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7258	2024-12-06T21:23:54.812Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7259	2024-12-06T21:23:54.817Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7260	2024-12-06T21:23:54.817Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7261	2024-12-06T21:23:54.820Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7262	2024-12-06T21:23:54.820Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7263	2024-12-06T21:23:54.824Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
7264	2024-12-06T21:23:54.824Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
7265	2024-12-06T21:23:54.824Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "b0491362-d292-4ba1-91f3-b6fd2a7cbb98", "date": "Fri, 06 Dec 2024 21:23:53 GMT"} })
7266	2024-12-06T21:23:54.825Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
7267	2024-12-06T21:23:54.825Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7268	2024-12-06T21:23:54.825Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d914d3ec-29a2-43f4-ba84-82103fb545ef", "content-length": "26", "date": "Fri, 06 Dec 2024 21:23:54 GMT"} })
7269	2024-12-06T21:23:54.825Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7270	2024-12-06T21:23:54.826Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 419.173616ms iteration = 19 status = {}
7271	2024-12-06T21:23:54.827Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
7272	2024-12-06T21:23:54.828Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
7273	2024-12-06T21:23:54.828Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7274	2024-12-06T21:23:54.828Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "890ea736-c18d-47d8-a2db-4face6a51efb", "content-length": "26", "date": "Fri, 06 Dec 2024 21:23:54 GMT"} })
7275	2024-12-06T21:23:54.828Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7276	2024-12-06T21:23:54.829Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
7277	2024-12-06T21:23:54.839Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7278	2024-12-06T21:23:54.841Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
7279	2024-12-06T21:23:54.841Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7280	2024-12-06T21:23:54.841Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7281	2024-12-06T21:23:54.846Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7282	2024-12-06T21:23:54.849Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
7283	2024-12-06T21:23:54.849Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7284	2024-12-06T21:23:54.849Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7285	2024-12-06T21:23:54.851Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 05c53cbc-f66c-491f-af8d-6f291fe21596 dns_zone_name = oxide.test found_so_far = 1 version = 2
7286	2024-12-06T21:23:54.851Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
7287	2024-12-06T21:23:54.851Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-06 21:15:07.045426 UTC dns_group = external generation = 2
7288	2024-12-06T21:23:54.851Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 444.771155ms iteration = 10 status = {"generation":2}
7289	2024-12-06T21:23:54.860Z	WARN	nexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
7290	2024-12-06T21:23:54.860Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 452.98827ms iteration = 10 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
7291	2024-12-06T21:23:54.869Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7292	2024-12-06T21:23:54.869Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7293	2024-12-06T21:23:54.871Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7294	2024-12-06T21:23:54.873Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7295	2024-12-06T21:23:54.873Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7296	2024-12-06T21:23:54.873Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7297	2024-12-06T21:23:54.874Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7298	2024-12-06T21:23:54.876Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7299	2024-12-06T21:23:54.876Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7300	2024-12-06T21:23:54.876Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7301	2024-12-06T21:23:54.876Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 469.765615ms iteration = 20 status = {"physical_disks_added":0}
7302	2024-12-06T21:23:54.894Z	DEBG	nexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-06T21:14:50.566199Z, time_modified: 2024-12-06T21:14:50.566199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, db_silo: Silo { identity: SiloIdentity { id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-06T21:15:06.991365Z, time_modified: 2024-12-06T21:15:06.991365Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, db_silo: Silo { identity: SiloIdentity { id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-06T21:15:06.991365Z, time_modified: 2024-12-06T21:15:06.991365Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1" }] }) }
7303	2024-12-06T21:23:54.894Z	DEBG	nexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 487.934353ms iteration = 10 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"683d02e1-6ec8-48bf-98d0-db6c94785f80","tls_certs":["d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1"]}},"default_endpoint":{"silo_id":"683d02e1-6ec8-48bf-98d0-db6c94785f80","tls_certs":["d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
7304	2024-12-06T21:23:54.907Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
7305	2024-12-06T21:23:54.907Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "a8ba7c74-26d4-4675-be99-b9363184d6b0", "date": "Fri, 06 Dec 2024 21:23:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
7306	2024-12-06T21:23:54.907Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
7307	2024-12-06T21:23:54.909Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "9a0f77e6-3179-4cf7-8e4b-34b13a3ec265", "date": "Fri, 06 Dec 2024 21:23:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
7308	2024-12-06T21:23:54.909Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
7309	2024-12-06T21:23:54.911Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "58c8a7a5-021c-4524-b5f8-ce4f3eaa821a", "date": "Fri, 06 Dec 2024 21:23:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
7310	2024-12-06T21:23:54.911Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
7311	2024-12-06T21:23:54.913Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
7312	2024-12-06T21:23:54.913Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 506.413866ms iteration = 12 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
7313	2024-12-06T21:23:54.971Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 564.027641ms iteration = 19 status = {}
7314	2024-12-06T21:23:55.075Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7315	2024-12-06T21:23:55.076Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7316	2024-12-06T21:23:55.078Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 3d56b7bf-53ed-4147-b45b-9cef154d7338, rack_id: 2d75e4a3-6363-4fd1-935e-989d31fd7d71, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(02859c9b-92d7-4b19-b425-8f266763be54) } rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
7317	2024-12-06T21:23:55.078Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7318	2024-12-06T21:23:55.078Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7319	2024-12-06T21:23:55.087Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
7320	2024-12-06T21:23:55.088Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a399328-a99e-4d07-82cc-7525a267dd2f", "content-length": "134", "date": "Fri, 06 Dec 2024 21:23:54 GMT"} })
7321	2024-12-06T21:23:55.088Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
7322	2024-12-06T21:23:55.088Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 02859c9b-92d7-4b19-b425-8f266763be54
7323	2024-12-06T21:23:55.088Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
7324	2024-12-06T21:23:55.088Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
7325	2024-12-06T21:23:55.088Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
7326	2024-12-06T21:23:55.088Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
7327	2024-12-06T21:23:55.088Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
7328	2024-12-06T21:23:55.088Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
7329	2024-12-06T21:23:55.088Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
7330	2024-12-06T21:23:55.088Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
7331	2024-12-06T21:23:55.089Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
7332	2024-12-06T21:23:55.089Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7333	2024-12-06T21:23:55.089Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7334	2024-12-06T21:23:55.091Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
7335	2024-12-06T21:23:55.091Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb61091f-d20d-47e5-847f-de56926990cb", "content-length": "2", "date": "Fri, 06 Dec 2024 21:23:54 GMT"} })
7336	2024-12-06T21:23:55.091Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
7337	2024-12-06T21:23:55.091Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "e74cc554-e43e-490a-9cdf-96b1cdcff2fd", "content-length": "52", "date": "Fri, 06 Dec 2024 21:23:54 GMT"} })
7338	2024-12-06T21:23:55.091Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
7339	2024-12-06T21:23:55.091Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
7340	2024-12-06T21:23:55.128Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
7341	2024-12-06T21:23:55.129Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "b259f35a-1c1d-477b-af3e-8949ddeabeba", "content-length": "1", "date": "Fri, 06 Dec 2024 21:23:54 GMT"} })
7342	2024-12-06T21:23:55.129Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
7343	2024-12-06T21:23:55.130Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "77d37fe0-fa29-42d7-bdfd-df71fad3b4e4", "date": "Fri, 06 Dec 2024 21:23:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
7344	2024-12-06T21:23:55.130Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
7345	2024-12-06T21:23:55.130Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7346	2024-12-06T21:23:55.130Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7347	2024-12-06T21:23:55.131Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "4ec4f3b7-a37e-4d07-9b47-9ae1c67f2f22", "date": "Fri, 06 Dec 2024 21:23:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
7348	2024-12-06T21:23:55.131Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
7349	2024-12-06T21:23:55.133Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 726.082809ms iteration = 19 status = null
7350	2024-12-06T21:23:55.133Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
7351	2024-12-06T21:23:55.133Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 726.949414ms iteration = 12 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
7352	2024-12-06T21:23:55.141Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
7353	2024-12-06T21:23:55.191Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "00f8c3d5-941a-4297-b559-3f096d6c81ac", "date": "Fri, 06 Dec 2024 21:23:55 GMT"} })
7354	2024-12-06T21:23:55.191Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch = Switch0
7355	2024-12-06T21:23:55.191Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
7356	2024-12-06T21:23:55.191Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "f2423524-711e-4de7-92ab-3e0b18e114ec", "content-length": "682", "date": "Fri, 06 Dec 2024 21:23:55 GMT"} })
7357	2024-12-06T21:23:55.191Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7358	2024-12-06T21:23:55.191Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7359	2024-12-06T21:23:55.195Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "5b04b0e1-6e9c-4ed7-8760-881a28d9c2fc", "content-length": "691", "date": "Fri, 06 Dec 2024 21:23:55 GMT"} })
7360	2024-12-06T21:23:55.195Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
7361	2024-12-06T21:23:55.195Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
7362	2024-12-06T21:23:55.195Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
7363	2024-12-06T21:23:55.195Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7364	2024-12-06T21:23:55.195Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7365	2024-12-06T21:23:55.197Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 790.511016ms iteration = 19 status = {}
7366	2024-12-06T21:23:55.199Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7367	2024-12-06T21:23:55.199Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7368	2024-12-06T21:23:55.201Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7369	2024-12-06T21:23:55.201Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7370	2024-12-06T21:23:55.209Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 802.164757ms iteration = 19 status = {}
7371	2024-12-06T21:23:56.541Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 265 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = cb8064ff-e80c-4665-9834-86138c399f6b response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
7372	2024-12-06T21:24:04.406Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 56 reason = Timeout
7373	2024-12-06T21:24:04.406Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 56 reason = Timeout
7374	2024-12-06T21:24:04.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7375	2024-12-06T21:24:04.406Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7376	2024-12-06T21:24:04.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7377	2024-12-06T21:24:04.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7378	2024-12-06T21:24:04.409Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7379	2024-12-06T21:24:04.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7380	2024-12-06T21:24:04.409Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7381	2024-12-06T21:24:04.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7382	2024-12-06T21:24:04.411Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7383	2024-12-06T21:24:04.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7384	2024-12-06T21:24:04.414Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7385	2024-12-06T21:24:04.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7386	2024-12-06T21:24:04.416Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7387	2024-12-06T21:24:04.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
7388	2024-12-06T21:24:04.418Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7389	2024-12-06T21:24:04.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7390	2024-12-06T21:24:04.428Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
7391	2024-12-06T21:24:04.428Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.307609ms iteration = 56 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
7392	2024-12-06T21:24:04.438Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.579478ms iteration = 56 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
7393	2024-12-06T21:24:06.541Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 264 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = f48fc3b8-9906-4392-bce3-c04641d444f4 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
7394	2024-12-06T21:24:14.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 57 reason = Timeout
7395	2024-12-06T21:24:14.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 57 reason = Timeout
7396	2024-12-06T21:24:14.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7397	2024-12-06T21:24:14.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7398	2024-12-06T21:24:14.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7399	2024-12-06T21:24:14.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7400	2024-12-06T21:24:14.410Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7401	2024-12-06T21:24:14.410Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7402	2024-12-06T21:24:14.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7403	2024-12-06T21:24:14.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7404	2024-12-06T21:24:14.412Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7405	2024-12-06T21:24:14.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7406	2024-12-06T21:24:14.414Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7407	2024-12-06T21:24:14.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7408	2024-12-06T21:24:14.415Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7409	2024-12-06T21:24:14.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
7410	2024-12-06T21:24:14.418Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7411	2024-12-06T21:24:14.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7412	2024-12-06T21:24:14.429Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
7413	2024-12-06T21:24:14.430Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.471759ms iteration = 57 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
7414	2024-12-06T21:24:14.439Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.644369ms iteration = 57 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
7415	2024-12-06T21:24:16.541Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 450 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 6132d91a-53cb-4c95-a7d6-300d9787eff7 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
7416	2024-12-06T21:24:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 20 reason = Timeout
7417	2024-12-06T21:24:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 20 reason = Timeout
7418	2024-12-06T21:24:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
7419	2024-12-06T21:24:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
7420	2024-12-06T21:24:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 20 reason = Timeout
7421	2024-12-06T21:24:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
7422	2024-12-06T21:24:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 20 reason = Timeout
7423	2024-12-06T21:24:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7424	2024-12-06T21:24:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 20 reason = Timeout
7425	2024-12-06T21:24:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 20 reason = Timeout
7426	2024-12-06T21:24:24.407Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
7427	2024-12-06T21:24:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7428	2024-12-06T21:24:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
7429	2024-12-06T21:24:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 20 reason = Timeout
7430	2024-12-06T21:24:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7431	2024-12-06T21:24:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 20 reason = Timeout
7432	2024-12-06T21:24:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
7433	2024-12-06T21:24:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 20 reason = Timeout
7434	2024-12-06T21:24:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
7435	2024-12-06T21:24:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 20 reason = Timeout
7436	2024-12-06T21:24:24.407Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
7437	2024-12-06T21:24:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7438	2024-12-06T21:24:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 20 reason = Timeout
7439	2024-12-06T21:24:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
7440	2024-12-06T21:24:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 20 reason = Timeout
7441	2024-12-06T21:24:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 21 reason = Timeout
7442	2024-12-06T21:24:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7443	2024-12-06T21:24:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 58 reason = Timeout
7444	2024-12-06T21:24:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7445	2024-12-06T21:24:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7446	2024-12-06T21:24:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 20 reason = Timeout
7447	2024-12-06T21:24:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 20 reason = Timeout
7448	2024-12-06T21:24:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
7449	2024-12-06T21:24:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 58 reason = Timeout
7450	2024-12-06T21:24:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7451	2024-12-06T21:24:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
7452	2024-12-06T21:24:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
7453	2024-12-06T21:24:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
7454	2024-12-06T21:24:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
7455	2024-12-06T21:24:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7456	2024-12-06T21:24:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
7457	2024-12-06T21:24:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
7458	2024-12-06T21:24:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7459	2024-12-06T21:24:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7460	2024-12-06T21:24:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7461	2024-12-06T21:24:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
7462	2024-12-06T21:24:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7463	2024-12-06T21:24:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7464	2024-12-06T21:24:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7465	2024-12-06T21:24:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7466	2024-12-06T21:24:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
7467	2024-12-06T21:24:24.410Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.949249ms iteration = 20 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
7468	2024-12-06T21:24:24.411Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7469	2024-12-06T21:24:24.412Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1379, tv_nsec: 909863854 } })
7470	2024-12-06T21:24:24.412Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
7471	2024-12-06T21:24:24.413Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 5.975358ms iteration = 20 status = {"errors":[],"records_set_to_done":[]}
7472	2024-12-06T21:24:24.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
7473	2024-12-06T21:24:24.413Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7474	2024-12-06T21:24:24.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7475	2024-12-06T21:24:24.415Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7476	2024-12-06T21:24:24.417Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
7477	2024-12-06T21:24:24.417Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7478	2024-12-06T21:24:24.417Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1379, tv_nsec: 915062590 } })
7479	2024-12-06T21:24:24.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7480	2024-12-06T21:24:24.417Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
7481	2024-12-06T21:24:24.419Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7482	2024-12-06T21:24:24.420Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1379, tv_nsec: 918612518 } })
7483	2024-12-06T21:24:24.421Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
7484	2024-12-06T21:24:24.422Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7485	2024-12-06T21:24:24.422Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7486	2024-12-06T21:24:24.422Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7487	2024-12-06T21:24:24.424Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7488	2024-12-06T21:24:24.425Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7489	2024-12-06T21:24:24.446Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7490	2024-12-06T21:24:24.447Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
7491	2024-12-06T21:24:24.447Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7492	2024-12-06T21:24:24.447Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
7493	2024-12-06T21:24:24.449Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 41.741295ms iteration = 58 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
7494	2024-12-06T21:24:24.451Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
7495	2024-12-06T21:24:24.451Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
7496	2024-12-06T21:24:24.452Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7497	2024-12-06T21:24:24.452Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7498	2024-12-06T21:24:24.453Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
7499	2024-12-06T21:24:24.453Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
7500	2024-12-06T21:24:24.454Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
7501	2024-12-06T21:24:24.454Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7502	2024-12-06T21:24:24.454Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
7503	2024-12-06T21:24:24.454Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7504	2024-12-06T21:24:24.454Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7505	2024-12-06T21:24:24.454Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
7506	2024-12-06T21:24:24.454Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7507	2024-12-06T21:24:24.455Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
7508	2024-12-06T21:24:24.456Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 48.537106ms iteration = 20 status = {"errors":[],"garbage_collect_requested":[]}
7509	2024-12-06T21:24:24.457Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7510	2024-12-06T21:24:24.457Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7511	2024-12-06T21:24:24.458Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
7512	2024-12-06T21:24:24.458Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
7513	2024-12-06T21:24:24.458Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
7514	2024-12-06T21:24:24.458Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
7515	2024-12-06T21:24:24.459Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7516	2024-12-06T21:24:24.459Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7517	2024-12-06T21:24:24.459Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7518	2024-12-06T21:24:24.461Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
7519	2024-12-06T21:24:24.461Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7520	2024-12-06T21:24:24.462Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
7521	2024-12-06T21:24:24.462Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
7522	2024-12-06T21:24:24.462Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7523	2024-12-06T21:24:24.462Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
7524	2024-12-06T21:24:24.463Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
7525	2024-12-06T21:24:24.463Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7526	2024-12-06T21:24:24.463Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7527	2024-12-06T21:24:24.469Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
7528	2024-12-06T21:24:24.469Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7529	2024-12-06T21:24:24.469Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d8432074-de78-4d03-8081-c143375e11cb", "content-length": "26", "date": "Fri, 06 Dec 2024 21:24:24 GMT"} })
7530	2024-12-06T21:24:24.469Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7531	2024-12-06T21:24:24.470Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7532	2024-12-06T21:24:24.470Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7533	2024-12-06T21:24:24.471Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
7534	2024-12-06T21:24:24.474Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
7535	2024-12-06T21:24:24.474Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 66.965218ms iteration = 20 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
7536	2024-12-06T21:24:24.476Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7537	2024-12-06T21:24:24.478Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
7538	2024-12-06T21:24:24.479Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7539	2024-12-06T21:24:24.479Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7540	2024-12-06T21:24:24.488Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 zone_address = fd00:1122:3344:101::2
7541	2024-12-06T21:24:24.488Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7542	2024-12-06T21:24:24.488Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0f176fda-8936-4e4e-80a1-a2c8b55637ee", "content-length": "26", "date": "Fri, 06 Dec 2024 21:24:24 GMT"} })
7543	2024-12-06T21:24:24.489Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7544	2024-12-06T21:24:24.490Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
7545	2024-12-06T21:24:24.490Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7546	2024-12-06T21:24:24.490Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 83.055512ms iteration = 20 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
7547	2024-12-06T21:24:24.490Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e5b8593-8766-44fa-ac4d-e5b6ecab5912", "content-length": "26", "date": "Fri, 06 Dec 2024 21:24:24 GMT"} })
7548	2024-12-06T21:24:24.491Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7549	2024-12-06T21:24:24.491Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
7550	2024-12-06T21:24:24.491Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
7551	2024-12-06T21:24:24.491Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 83.974528ms iteration = 58 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
7552	2024-12-06T21:24:24.492Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
7553	2024-12-06T21:24:24.492Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 85.458271ms iteration = 20 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
7554	2024-12-06T21:24:24.494Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7555	2024-12-06T21:24:24.494Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7556	2024-12-06T21:24:24.496Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7557	2024-12-06T21:24:24.496Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7558	2024-12-06T21:24:24.496Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
7559	2024-12-06T21:24:24.496Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
7560	2024-12-06T21:24:24.496Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
7561	2024-12-06T21:24:24.497Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
7562	2024-12-06T21:24:24.497Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 90.197479ms iteration = 20 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
7563	2024-12-06T21:24:24.498Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7564	2024-12-06T21:24:24.500Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cbc5c1c6-5668-455a-85ef-94f4aebe0f80, lookup_type: ById(cbc5c1c6-5668-455a-85ef-94f4aebe0f80) } result = Ok(())
7565	2024-12-06T21:24:24.500Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7566	2024-12-06T21:24:24.500Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7567	2024-12-06T21:24:24.502Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 94.902416ms iteration = 20 status = 0
7568	2024-12-06T21:24:24.528Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7569	2024-12-06T21:24:24.528Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7570	2024-12-06T21:24:24.528Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
7571	2024-12-06T21:24:24.528Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
7572	2024-12-06T21:24:24.528Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 121.220109ms iteration = 20 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
7573	2024-12-06T21:24:24.555Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 147.370107ms iteration = 21 status = {"physical_disks_added":0}
7574	2024-12-06T21:24:24.615Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
7575	2024-12-06T21:24:24.616Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "8817643b-efbf-4cfe-96d0-51e78aa2829e", "content-length": "1", "date": "Fri, 06 Dec 2024 21:24:23 GMT"} })
7576	2024-12-06T21:24:24.618Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7577	2024-12-06T21:24:24.618Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7578	2024-12-06T21:24:24.620Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 213.179757ms iteration = 20 status = null
7579	2024-12-06T21:24:24.635Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7580	2024-12-06T21:24:24.635Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7581	2024-12-06T21:24:24.637Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7582	2024-12-06T21:24:24.640Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7583	2024-12-06T21:24:24.640Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7584	2024-12-06T21:24:24.641Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7585	2024-12-06T21:24:24.654Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
7586	2024-12-06T21:24:24.654Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
7587	2024-12-06T21:24:24.655Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "0182570b-7150-4b33-9e8c-36f209651d54", "content-length": "372", "date": "Fri, 06 Dec 2024 21:24:23 GMT"} })
7588	2024-12-06T21:24:24.655Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7589	2024-12-06T21:24:24.655Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7590	2024-12-06T21:24:24.665Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 257.89087ms iteration = 20 status = {}
7591	2024-12-06T21:24:24.698Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7592	2024-12-06T21:24:24.700Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7593	2024-12-06T21:24:24.700Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7594	2024-12-06T21:24:24.700Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7595	2024-12-06T21:24:24.703Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
7596	2024-12-06T21:24:24.703Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
7597	2024-12-06T21:24:24.703Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "87a98989-bb83-4a00-bcc1-c5099a2ef372", "date": "Fri, 06 Dec 2024 21:24:23 GMT"} })
7598	2024-12-06T21:24:24.703Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7599	2024-12-06T21:24:24.704Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7600	2024-12-06T21:24:24.705Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7601	2024-12-06T21:24:24.706Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7602	2024-12-06T21:24:24.708Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7603	2024-12-06T21:24:24.709Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7604	2024-12-06T21:24:24.712Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
7605	2024-12-06T21:24:24.712Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
7606	2024-12-06T21:24:24.712Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "69798f10-838d-44b5-8d22-75402f043fb6", "date": "Fri, 06 Dec 2024 21:24:23 GMT"} })
7607	2024-12-06T21:24:24.714Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 306.716381ms iteration = 20 status = {}
7608	2024-12-06T21:24:24.764Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7609	2024-12-06T21:24:24.765Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7610	2024-12-06T21:24:24.766Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
7611	2024-12-06T21:24:24.766Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "a59d93c5-d134-43cd-b567-6b692d9faca8", "content-length": "691", "date": "Fri, 06 Dec 2024 21:24:24 GMT"} })
7612	2024-12-06T21:24:24.766Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
7613	2024-12-06T21:24:24.766Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
7614	2024-12-06T21:24:24.766Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
7615	2024-12-06T21:24:24.767Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 3d56b7bf-53ed-4147-b45b-9cef154d7338, rack_id: 2d75e4a3-6363-4fd1-935e-989d31fd7d71, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(02859c9b-92d7-4b19-b425-8f266763be54) } rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
7616	2024-12-06T21:24:24.767Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7617	2024-12-06T21:24:24.767Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7618	2024-12-06T21:24:24.768Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 360.737106ms iteration = 20 status = {}
7619	2024-12-06T21:24:24.776Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
7620	2024-12-06T21:24:24.777Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c71f473-73e5-4ff6-b325-962b29ed27fb", "content-length": "134", "date": "Fri, 06 Dec 2024 21:24:24 GMT"} })
7621	2024-12-06T21:24:24.777Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
7622	2024-12-06T21:24:24.777Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 02859c9b-92d7-4b19-b425-8f266763be54
7623	2024-12-06T21:24:24.777Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
7624	2024-12-06T21:24:24.777Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
7625	2024-12-06T21:24:24.777Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
7626	2024-12-06T21:24:24.777Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
7627	2024-12-06T21:24:24.777Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
7628	2024-12-06T21:24:24.777Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
7629	2024-12-06T21:24:24.777Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
7630	2024-12-06T21:24:24.777Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
7631	2024-12-06T21:24:24.777Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
7632	2024-12-06T21:24:24.777Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7633	2024-12-06T21:24:24.778Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7634	2024-12-06T21:24:24.779Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
7635	2024-12-06T21:24:24.779Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "6dedf3aa-00b6-49e0-91fa-91e2ca5fa0fe", "content-length": "2", "date": "Fri, 06 Dec 2024 21:24:24 GMT"} })
7636	2024-12-06T21:24:24.779Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
7637	2024-12-06T21:24:24.779Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "70331d57-aa88-4f0d-9806-757a88b4a5c8", "content-length": "52", "date": "Fri, 06 Dec 2024 21:24:24 GMT"} })
7638	2024-12-06T21:24:24.780Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
7639	2024-12-06T21:24:24.780Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
7640	2024-12-06T21:24:24.877Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "0553e6b0-fbf8-45bc-882f-ed29d54164ff", "date": "Fri, 06 Dec 2024 21:24:24 GMT"} })
7641	2024-12-06T21:24:24.877Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch = Switch0
7642	2024-12-06T21:24:24.877Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
7643	2024-12-06T21:24:24.878Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "96ea1315-c840-4232-a358-d10024d992f2", "content-length": "682", "date": "Fri, 06 Dec 2024 21:24:24 GMT"} })
7644	2024-12-06T21:24:24.878Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7645	2024-12-06T21:24:24.878Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7646	2024-12-06T21:24:24.882Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7647	2024-12-06T21:24:24.882Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7648	2024-12-06T21:24:24.885Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7649	2024-12-06T21:24:24.885Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7650	2024-12-06T21:24:24.887Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7651	2024-12-06T21:24:24.887Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7652	2024-12-06T21:24:24.894Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 487.081259ms iteration = 20 status = {}
7653	2024-12-06T21:24:26.541Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 293 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 0c8315db-4996-4f01-8c7c-b93f46b1e5f9 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
7654	2024-12-06T21:24:34.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 59 reason = Timeout
7655	2024-12-06T21:24:34.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7656	2024-12-06T21:24:34.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 59 reason = Timeout
7657	2024-12-06T21:24:34.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7658	2024-12-06T21:24:34.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7659	2024-12-06T21:24:34.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7660	2024-12-06T21:24:34.409Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7661	2024-12-06T21:24:34.409Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7662	2024-12-06T21:24:34.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7663	2024-12-06T21:24:34.411Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.471723ms iteration = 59 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
7664	2024-12-06T21:24:34.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7665	2024-12-06T21:24:34.411Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7666	2024-12-06T21:24:34.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7667	2024-12-06T21:24:34.416Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7668	2024-12-06T21:24:34.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7669	2024-12-06T21:24:34.418Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7670	2024-12-06T21:24:34.420Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
7671	2024-12-06T21:24:34.420Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7672	2024-12-06T21:24:34.420Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7673	2024-12-06T21:24:34.430Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
7674	2024-12-06T21:24:34.430Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.315622ms iteration = 59 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
7675	2024-12-06T21:24:36.541Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 268 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 82a1117f-ebee-4206-a831-3c42ab631141 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
7676	2024-12-06T21:24:44.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 60 reason = Timeout
7677	2024-12-06T21:24:44.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 60 reason = Timeout
7678	2024-12-06T21:24:44.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7679	2024-12-06T21:24:44.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7680	2024-12-06T21:24:44.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7681	2024-12-06T21:24:44.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7682	2024-12-06T21:24:44.409Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7683	2024-12-06T21:24:44.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7684	2024-12-06T21:24:44.410Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7685	2024-12-06T21:24:44.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7686	2024-12-06T21:24:44.412Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7687	2024-12-06T21:24:44.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7688	2024-12-06T21:24:44.415Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7689	2024-12-06T21:24:44.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7690	2024-12-06T21:24:44.416Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7691	2024-12-06T21:24:44.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
7692	2024-12-06T21:24:44.419Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7693	2024-12-06T21:24:44.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7694	2024-12-06T21:24:44.431Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
7695	2024-12-06T21:24:44.431Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.523376ms iteration = 60 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
7696	2024-12-06T21:24:44.438Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.156312ms iteration = 60 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
7697	2024-12-06T21:24:46.542Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 324 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 99d47faa-cf22-4632-bfef-790989d6c00d response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
7698	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 61 reason = Timeout
7699	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7700	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 21 reason = Timeout
7701	2024-12-06T21:24:54.407Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
7702	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 11 reason = Timeout
7703	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7704	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 13 reason = Timeout
7705	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 21 reason = Timeout
7706	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
7707	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
7708	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = external_endpoints iteration = 11 reason = Timeout
7709	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 21 reason = Timeout
7710	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7711	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
7712	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 21 reason = Timeout
7713	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 11 reason = Timeout
7714	2024-12-06T21:24:54.407Z	INFO	nexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
7715	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
7716	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = inventory_collection iteration = 2 reason = Timeout
7717	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 11 reason = Timeout
7718	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
7719	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
7720	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
7721	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 21 reason = Timeout
7722	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7723	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7724	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 21 reason = Timeout
7725	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 21 reason = Timeout
7726	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7727	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
7728	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 11 reason = Timeout
7729	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 61 reason = Timeout
7730	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 11 reason = Timeout
7731	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7732	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
7733	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 21 reason = Timeout
7734	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 21 reason = Timeout
7735	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
7736	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7737	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 21 reason = Timeout
7738	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
7739	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 21 reason = Timeout
7740	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 11 reason = Timeout
7741	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7742	2024-12-06T21:24:54.407Z	INFO	nexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-06 21:14:54.407420681 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
7743	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
7744	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 11 reason = Timeout
7745	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
7746	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 21 reason = Timeout
7747	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
7748	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 13 reason = Timeout
7749	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 12 reason = Timeout
7750	2024-12-06T21:24:54.407Z	WARN	nexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0
7751	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 21 reason = Timeout
7752	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = saga_recovery iteration = 2 reason = Timeout
7753	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 15.6\u{b5}s iteration = 12 status = {"enabled":false,"target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0"}
7754	2024-12-06T21:24:54.407Z	DEBG	nexus (SagaRecoverer): listing candidate sagas for recovery
7755	2024-12-06T21:24:54.407Z	DEBG	nexus (SagaRecoverer): roles roles = RoleSet { roles: {} }
7756	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 22 reason = Timeout
7757	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 21 reason = Timeout
7758	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7759	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
7760	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 11 reason = Timeout
7761	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
7762	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 11 reason = Timeout
7763	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7764	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
7765	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 3 reason = Timeout
7766	2024-12-06T21:24:54.407Z	DEBG	nexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
7767	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7768	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
7769	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7770	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7771	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
7772	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7773	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
7774	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
7775	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
7776	2024-12-06T21:24:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7777	2024-12-06T21:24:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7778	2024-12-06T21:24:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
7779	2024-12-06T21:24:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
7780	2024-12-06T21:24:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7781	2024-12-06T21:24:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
7782	2024-12-06T21:24:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7783	2024-12-06T21:24:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7784	2024-12-06T21:24:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
7785	2024-12-06T21:24:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7786	2024-12-06T21:24:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7787	2024-12-06T21:24:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
7788	2024-12-06T21:24:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
7789	2024-12-06T21:24:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7790	2024-12-06T21:24:54.408Z	DEBG	nexus (SagaRecoverer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a, .. }) resource = Database result = Ok(())
7791	2024-12-06T21:24:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
7792	2024-12-06T21:24:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
7793	2024-12-06T21:24:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7794	2024-12-06T21:24:54.408Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("709da000-e0da-4b29-8160-3112fd080927.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a03854a6-7a78-402b-93b4-65c6d36bf236.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a03854a6-7a78-402b-93b4-65c6d36bf236.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1409, tv_nsec: 905861053 } })
7795	2024-12-06T21:24:54.408Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4aff1ae3-2d71-40bf-bc46-79a7276da3c2.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4dccdb26-9cf9-40aa-9229-5d35905cf0e1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("79d475dc-44ec-44f4-9e51-ea8a753c39f3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("79d475dc-44ec-44f4-9e51-ea8a753c39f3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1409, tv_nsec: 905861683 } })
7796	2024-12-06T21:24:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
7797	2024-12-06T21:24:54.408Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
7798	2024-12-06T21:24:54.408Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.379903ms iteration = 11 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
7799	2024-12-06T21:24:54.408Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
7800	2024-12-06T21:24:54.408Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.686218ms iteration = 11 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
7801	2024-12-06T21:24:54.410Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7802	2024-12-06T21:24:54.411Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.936435ms iteration = 21 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
7803	2024-12-06T21:24:54.412Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
7804	2024-12-06T21:24:54.412Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7805	2024-12-06T21:24:54.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7806	2024-12-06T21:24:54.414Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7807	2024-12-06T21:24:54.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7808	2024-12-06T21:24:54.415Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7809	2024-12-06T21:24:54.415Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7810	2024-12-06T21:24:54.416Z	DEBG	nexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 9.525156ms iteration = 11 status = {"errors":[],"found_port_ok":[]}
7811	2024-12-06T21:24:54.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
7812	2024-12-06T21:24:54.417Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
7813	2024-12-06T21:24:54.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
7814	2024-12-06T21:24:54.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
7815	2024-12-06T21:24:54.418Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7816	2024-12-06T21:24:54.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7817	2024-12-06T21:24:54.419Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7818	2024-12-06T21:24:54.419Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7819	2024-12-06T21:24:54.420Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7820	2024-12-06T21:24:54.421Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1409, tv_nsec: 919008998 } })
7821	2024-12-06T21:24:54.421Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
7822	2024-12-06T21:24:54.422Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
7823	2024-12-06T21:24:54.422Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
7824	2024-12-06T21:24:54.422Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
7825	2024-12-06T21:24:54.422Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7826	2024-12-06T21:24:54.422Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7827	2024-12-06T21:24:54.422Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7828	2024-12-06T21:24:54.423Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
7829	2024-12-06T21:24:54.423Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7830	2024-12-06T21:24:54.423Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7831	2024-12-06T21:24:54.424Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7832	2024-12-06T21:24:54.424Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7833	2024-12-06T21:24:54.432Z	INFO	nexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-06 21:14:54.407420681 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
7834	2024-12-06T21:24:54.432Z	DEBG	nexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 24.640334ms iteration = 11 status = {"expiration":"2024-12-06T21:14:54.407420681Z","pruned":[]}
7835	2024-12-06T21:24:54.432Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1409, tv_nsec: 930470426 } })
7836	2024-12-06T21:24:54.433Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
7837	2024-12-06T21:24:54.435Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7838	2024-12-06T21:24:54.436Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7839	2024-12-06T21:24:54.438Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7840	2024-12-06T21:24:54.439Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7841	2024-12-06T21:24:54.440Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
7842	2024-12-06T21:24:54.440Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7843	2024-12-06T21:24:54.440Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7844	2024-12-06T21:24:54.440Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1409, tv_nsec: 938615050 } })
7845	2024-12-06T21:24:54.441Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
7846	2024-12-06T21:24:54.444Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
7847	2024-12-06T21:24:54.446Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7848	2024-12-06T21:24:54.446Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7849	2024-12-06T21:24:54.448Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7850	2024-12-06T21:24:54.449Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7851	2024-12-06T21:24:54.455Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7852	2024-12-06T21:24:54.457Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
7853	2024-12-06T21:24:54.458Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7854	2024-12-06T21:24:54.458Z	DEBG	nexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7855	2024-12-06T21:24:54.458Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7856	2024-12-06T21:24:54.458Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
7857	2024-12-06T21:24:54.458Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
7858	2024-12-06T21:24:54.458Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
7859	2024-12-06T21:24:54.459Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
7860	2024-12-06T21:24:54.460Z	DEBG	nexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
7861	2024-12-06T21:24:54.460Z	DEBG	nexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
7862	2024-12-06T21:24:54.461Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
7863	2024-12-06T21:24:54.462Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7864	2024-12-06T21:24:54.462Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7865	2024-12-06T21:24:54.463Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7866	2024-12-06T21:24:54.463Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7867	2024-12-06T21:24:54.463Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7868	2024-12-06T21:24:54.464Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7869	2024-12-06T21:24:54.466Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
7870	2024-12-06T21:24:54.466Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
7871	2024-12-06T21:24:54.467Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
7872	2024-12-06T21:24:54.468Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
7873	2024-12-06T21:24:54.468Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7874	2024-12-06T21:24:54.468Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7875	2024-12-06T21:24:54.468Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
7876	2024-12-06T21:24:54.468Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
7877	2024-12-06T21:24:54.468Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
7878	2024-12-06T21:24:54.468Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7879	2024-12-06T21:24:54.469Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7880	2024-12-06T21:24:54.471Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7881	2024-12-06T21:24:54.471Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7882	2024-12-06T21:24:54.471Z	DEBG	nexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
7883	2024-12-06T21:24:54.471Z	INFO	nexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
7884	2024-12-06T21:24:54.471Z	DEBG	nexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 64.257534ms iteration = 11 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
7885	2024-12-06T21:24:54.472Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 65.177789ms iteration = 61 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
7886	2024-12-06T21:24:54.472Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
7887	2024-12-06T21:24:54.473Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
7888	2024-12-06T21:24:54.473Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7889	2024-12-06T21:24:54.474Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7890	2024-12-06T21:24:54.475Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7891	2024-12-06T21:24:54.475Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 67.775072ms iteration = 21 status = {"errors":[],"garbage_collect_requested":[]}
7892	2024-12-06T21:24:54.475Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7893	2024-12-06T21:24:54.478Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7894	2024-12-06T21:24:54.478Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 71.773337ms iteration = 21 status = {"errors":[],"records_set_to_done":[]}
7895	2024-12-06T21:24:54.479Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
7896	2024-12-06T21:24:54.479Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 72.252496ms iteration = 21 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
7897	2024-12-06T21:24:54.479Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
7898	2024-12-06T21:24:54.479Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7899	2024-12-06T21:24:54.480Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
7900	2024-12-06T21:24:54.480Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7901	2024-12-06T21:24:54.480Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7902	2024-12-06T21:24:54.480Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7903	2024-12-06T21:24:54.481Z	INFO	nexus (SagaRecoverer): listed in-progress sagas count = 0 file = nexus/src/app/background/tasks/saga_recovery.rs:438
7904	2024-12-06T21:24:54.481Z	DEBG	nexus (BackgroundTasks): activation complete background_task = saga_recovery elapsed = 73.553556ms iteration = 2 status = {"last_pass":{"Success":{"nfailed":0,"nfound":0,"nrecovered":0,"nremoved":0,"nskipped":0}},"ntotal_failures":0,"ntotal_finished":0,"ntotal_recovered":0,"ntotal_sec_errors_bad_state":0,"ntotal_sec_errors_missing":0,"ntotal_started":0,"recent_failures":[],"recent_recoveries":[]}
7905	2024-12-06T21:24:54.481Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
7906	2024-12-06T21:24:54.481Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7907	2024-12-06T21:24:54.481Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7908	2024-12-06T21:24:54.481Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 74.334939ms iteration = 21 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
7909	2024-12-06T21:24:54.482Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7910	2024-12-06T21:24:54.483Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7911	2024-12-06T21:24:54.483Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
7912	2024-12-06T21:24:54.483Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 76.336162ms iteration = 61 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
7913	2024-12-06T21:24:54.484Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7914	2024-12-06T21:24:54.485Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7915	2024-12-06T21:24:54.485Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7916	2024-12-06T21:24:54.485Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7917	2024-12-06T21:24:54.486Z	DEBG	nexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
7918	2024-12-06T21:24:54.486Z	INFO	nexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
7919	2024-12-06T21:24:54.486Z	DEBG	nexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 79.565555ms iteration = 11 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
7920	2024-12-06T21:24:54.487Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7921	2024-12-06T21:24:54.487Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
7922	2024-12-06T21:24:54.487Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
7923	2024-12-06T21:24:54.487Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
7924	2024-12-06T21:24:54.487Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7925	2024-12-06T21:24:54.487Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7926	2024-12-06T21:24:54.489Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7927	2024-12-06T21:24:54.489Z	DEBG	nexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(9f79d131-32b9-4993-a220-b2cbdd58b99c, 0), (402d8dbb-782c-4c7c-8562-4644244a884f, 0), (b32b0640-ce0c-473b-803c-7574008a2e50, 0), (e5be7205-9f3f-4e66-bf37-506697e7f060, 0)] collection_id = 9f79d131-32b9-4993-a220-b2cbdd58b99c
7928	2024-12-06T21:24:54.489Z	DEBG	nexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
7929	2024-12-06T21:24:54.489Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
7930	2024-12-06T21:24:54.490Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
7931	2024-12-06T21:24:54.490Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7932	2024-12-06T21:24:54.490Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
7933	2024-12-06T21:24:54.490Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7934	2024-12-06T21:24:54.490Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
7935	2024-12-06T21:24:54.492Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
7936	2024-12-06T21:24:54.492Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7937	2024-12-06T21:24:54.492Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7938	2024-12-06T21:24:54.504Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
7939	2024-12-06T21:24:54.504Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
7940	2024-12-06T21:24:54.505Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7941	2024-12-06T21:24:54.505Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7942	2024-12-06T21:24:54.507Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7943	2024-12-06T21:24:54.507Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7944	2024-12-06T21:24:54.509Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
7945	2024-12-06T21:24:54.509Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
7946	2024-12-06T21:24:54.515Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
7947	2024-12-06T21:24:54.515Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
7948	2024-12-06T21:24:54.515Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
7949	2024-12-06T21:24:54.515Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
7950	2024-12-06T21:24:54.520Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7951	2024-12-06T21:24:54.520Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7952	2024-12-06T21:24:54.523Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7953	2024-12-06T21:24:54.523Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
7954	2024-12-06T21:24:54.525Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
7955	2024-12-06T21:24:54.525Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7956	2024-12-06T21:24:54.525Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7957	2024-12-06T21:24:54.535Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
7958	2024-12-06T21:24:54.535Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 93c3f865-bc00-42df-bef7-20e39342f84f dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
7959	2024-12-06T21:24:54.535Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7960	2024-12-06T21:24:54.535Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7961	2024-12-06T21:24:54.535Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7962	2024-12-06T21:24:54.538Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7963	2024-12-06T21:24:54.539Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
7964	2024-12-06T21:24:54.539Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7965	2024-12-06T21:24:54.539Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7966	2024-12-06T21:24:54.541Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7967	2024-12-06T21:24:54.541Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7968	2024-12-06T21:24:54.542Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
7969	2024-12-06T21:24:54.542Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7970	2024-12-06T21:24:54.542Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7971	2024-12-06T21:24:54.543Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7972	2024-12-06T21:24:54.543Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7973	2024-12-06T21:24:54.545Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 93c3f865-bc00-42df-bef7-20e39342f84f dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
7974	2024-12-06T21:24:54.545Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
7975	2024-12-06T21:24:54.545Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-06 21:15:06.025861 UTC dns_group = internal generation = 1
7976	2024-12-06T21:24:54.545Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 137.790931ms iteration = 11 status = {"generation":1}
7977	2024-12-06T21:24:54.545Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7978	2024-12-06T21:24:54.545Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 138.333319ms iteration = 21 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
7979	2024-12-06T21:24:54.549Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
7980	2024-12-06T21:24:54.549Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7981	2024-12-06T21:24:54.550Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7982	2024-12-06T21:24:54.552Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
7983	2024-12-06T21:24:54.552Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 05c53cbc-f66c-491f-af8d-6f291fe21596 dns_zone_name = oxide.test found_so_far = 0 version = 2
7984	2024-12-06T21:24:54.552Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7985	2024-12-06T21:24:54.553Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7986	2024-12-06T21:24:54.561Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7987	2024-12-06T21:24:54.564Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7988	2024-12-06T21:24:54.564Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7989	2024-12-06T21:24:54.565Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cbc5c1c6-5668-455a-85ef-94f4aebe0f80, lookup_type: ById(cbc5c1c6-5668-455a-85ef-94f4aebe0f80) } result = Ok(())
7990	2024-12-06T21:24:54.565Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7991	2024-12-06T21:24:54.565Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7992	2024-12-06T21:24:54.566Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 159.034319ms iteration = 21 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
7993	2024-12-06T21:24:54.571Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
7994	2024-12-06T21:24:54.571Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
7995	2024-12-06T21:24:54.571Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 164.128052ms iteration = 21 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
7996	2024-12-06T21:24:54.574Z	INFO	nexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 9f79d131-32b9-4993-a220-b2cbdd58b99c file = nexus/db-queries/src/db/datastore/inventory.rs:1442 ncabooses = 12 nclickhouse_keeper_membership = 0 ncollections = 1 ndatasets = 0 nerrors = 0 nnics = 0 nnvme_disk_firmware = 0 nphysical_disks = 0 nrot_pages = 8 nrots = 2 nsled_agent_zones = 0 nsled_agents = 0 nsps = 2 nzones = 0 nzpools = 0
7997	2024-12-06T21:24:54.574Z	DEBG	nexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
7998	2024-12-06T21:24:54.574Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
7999	2024-12-06T21:24:54.580Z	DEBG	nexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-06T21:14:50.566199Z, time_modified: 2024-12-06T21:14:50.566199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, db_silo: Silo { identity: SiloIdentity { id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-06T21:15:06.991365Z, time_modified: 2024-12-06T21:15:06.991365Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, db_silo: Silo { identity: SiloIdentity { id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-06T21:15:06.991365Z, time_modified: 2024-12-06T21:15:06.991365Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1" }] }) }
8000	2024-12-06T21:24:54.580Z	DEBG	nexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 173.328103ms iteration = 11 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"683d02e1-6ec8-48bf-98d0-db6c94785f80","tls_certs":["d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1"]}},"default_endpoint":{"silo_id":"683d02e1-6ec8-48bf-98d0-db6c94785f80","tls_certs":["d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
8001	2024-12-06T21:24:54.584Z	WARN	nexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
8002	2024-12-06T21:24:54.584Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 176.740949ms iteration = 11 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
8003	2024-12-06T21:24:54.604Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 197.169594ms iteration = 22 status = {"physical_disks_added":0}
8004	2024-12-06T21:24:54.627Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
8005	2024-12-06T21:24:54.680Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
8006	2024-12-06T21:24:54.680Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
8007	2024-12-06T21:24:54.680Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c4846f8a-e834-4800-a767-3b293d5f0c4f", "content-length": "26", "date": "Fri, 06 Dec 2024 21:24:53 GMT"} })
8008	2024-12-06T21:24:54.680Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
8009	2024-12-06T21:24:54.681Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
8010	2024-12-06T21:24:54.683Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
8011	2024-12-06T21:24:54.725Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 zone_address = fd00:1122:3344:101::2
8012	2024-12-06T21:24:54.725Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
8013	2024-12-06T21:24:54.726Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "39c5f011-2f79-4429-9d86-9769e257eb98", "content-length": "26", "date": "Fri, 06 Dec 2024 21:24:53 GMT"} })
8014	2024-12-06T21:24:54.726Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
8015	2024-12-06T21:24:54.728Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
8016	2024-12-06T21:24:54.728Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
8017	2024-12-06T21:24:54.728Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
8018	2024-12-06T21:24:54.729Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "f769eed6-18e3-4086-9a83-fc7d6a5bb916", "content-length": "372", "date": "Fri, 06 Dec 2024 21:24:53 GMT"} })
8019	2024-12-06T21:24:54.729Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8020	2024-12-06T21:24:54.729Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8021	2024-12-06T21:24:54.731Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8022	2024-12-06T21:24:54.731Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8023	2024-12-06T21:24:54.731Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8024	2024-12-06T21:24:54.732Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 325.258815ms iteration = 21 status = 0
8025	2024-12-06T21:24:54.733Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
8026	2024-12-06T21:24:54.734Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8027	2024-12-06T21:24:54.734Z	DEBG	nexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(402d8dbb-782c-4c7c-8562-4644244a884f, 0), (b32b0640-ce0c-473b-803c-7574008a2e50, 0), (e5be7205-9f3f-4e66-bf37-506697e7f060, 0)]
8028	2024-12-06T21:24:54.734Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8029	2024-12-06T21:24:54.734Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8030	2024-12-06T21:24:54.734Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1410, tv_nsec: 232203395 } })
8031	2024-12-06T21:24:54.734Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8032	2024-12-06T21:24:54.735Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 05c53cbc-f66c-491f-af8d-6f291fe21596 dns_zone_name = oxide.test found_so_far = 1 version = 2
8033	2024-12-06T21:24:54.735Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
8034	2024-12-06T21:24:54.735Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-06 21:15:07.045426 UTC dns_group = external generation = 2
8035	2024-12-06T21:24:54.736Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 328.738572ms iteration = 11 status = {"generation":2}
8036	2024-12-06T21:24:54.736Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8037	2024-12-06T21:24:54.736Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8038	2024-12-06T21:24:54.738Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8039	2024-12-06T21:24:54.738Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8040	2024-12-06T21:24:54.740Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8041	2024-12-06T21:24:54.740Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8042	2024-12-06T21:24:54.741Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8043	2024-12-06T21:24:54.742Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
8044	2024-12-06T21:24:54.742Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
8045	2024-12-06T21:24:54.742Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8046	2024-12-06T21:24:54.742Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "64450323-de03-4cfb-863a-28a893ee024d", "content-length": "26", "date": "Fri, 06 Dec 2024 21:24:53 GMT"} })
8047	2024-12-06T21:24:54.742Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
8048	2024-12-06T21:24:54.743Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8049	2024-12-06T21:24:54.744Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
8050	2024-12-06T21:24:54.747Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
8051	2024-12-06T21:24:54.747Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8052	2024-12-06T21:24:54.747Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8053	2024-12-06T21:24:54.749Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8054	2024-12-06T21:24:54.749Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8055	2024-12-06T21:24:54.751Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8056	2024-12-06T21:24:54.751Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8057	2024-12-06T21:24:54.753Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8058	2024-12-06T21:24:54.753Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8059	2024-12-06T21:24:54.755Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8060	2024-12-06T21:24:54.755Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8061	2024-12-06T21:24:54.757Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8062	2024-12-06T21:24:54.757Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8063	2024-12-06T21:24:54.758Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8064	2024-12-06T21:24:54.759Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8065	2024-12-06T21:24:54.760Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8066	2024-12-06T21:24:54.764Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
8067	2024-12-06T21:24:54.764Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8068	2024-12-06T21:24:54.764Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8069	2024-12-06T21:24:54.767Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8070	2024-12-06T21:24:54.767Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8071	2024-12-06T21:24:54.768Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8072	2024-12-06T21:24:54.769Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8073	2024-12-06T21:24:54.771Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8074	2024-12-06T21:24:54.774Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
8075	2024-12-06T21:24:54.774Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8076	2024-12-06T21:24:54.774Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8077	2024-12-06T21:24:54.779Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8078	2024-12-06T21:24:54.779Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8079	2024-12-06T21:24:54.781Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8080	2024-12-06T21:24:54.781Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8081	2024-12-06T21:24:54.782Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
8082	2024-12-06T21:24:54.782Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
8083	2024-12-06T21:24:54.782Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "738db55c-e5ab-4b8a-a6d9-f944779d1ad4", "date": "Fri, 06 Dec 2024 21:24:53 GMT"} })
8084	2024-12-06T21:24:54.782Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8085	2024-12-06T21:24:54.782Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8086	2024-12-06T21:24:54.783Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8087	2024-12-06T21:24:54.783Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8088	2024-12-06T21:24:54.785Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8089	2024-12-06T21:24:54.785Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8090	2024-12-06T21:24:54.785Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8091	2024-12-06T21:24:54.785Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8092	2024-12-06T21:24:54.788Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8093	2024-12-06T21:24:54.788Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8094	2024-12-06T21:24:54.788Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8095	2024-12-06T21:24:54.788Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8096	2024-12-06T21:24:54.790Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8097	2024-12-06T21:24:54.790Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8098	2024-12-06T21:24:54.791Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
8099	2024-12-06T21:24:54.791Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
8100	2024-12-06T21:24:54.791Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8101	2024-12-06T21:24:54.792Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "4444edae-e932-4b61-8670-eb0b6f672ef0", "date": "Fri, 06 Dec 2024 21:24:53 GMT"} })
8102	2024-12-06T21:24:54.792Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8103	2024-12-06T21:24:54.793Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8104	2024-12-06T21:24:54.794Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 387.051048ms iteration = 21 status = {}
8105	2024-12-06T21:24:54.799Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
8106	2024-12-06T21:24:54.799Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8107	2024-12-06T21:24:54.799Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8108	2024-12-06T21:24:54.800Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8109	2024-12-06T21:24:54.800Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8110	2024-12-06T21:24:54.802Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8111	2024-12-06T21:24:54.802Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8112	2024-12-06T21:24:54.803Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8113	2024-12-06T21:24:54.807Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
8114	2024-12-06T21:24:54.807Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8115	2024-12-06T21:24:54.807Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8116	2024-12-06T21:24:54.812Z	DEBG	nexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 112d04a8-d7b2-41ba-a8a9-b1b05b82789a, kind: Service { id: e8725c8c-13ab-4916-bf16-bb7f832014d5 }, name: Name("nexus-e8725c8c-13ab-4916-bf16-bb7f832014d5"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 245, 172])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 3b455a5c-72a6-4c48-bc31-0f3337db4efa, kind: Service { id: a211d4f7-7350-4466-97ca-36b58597b187 }, name: Name("nexus-a211d4f7-7350-4466-97ca-36b58597b187"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 181, 41])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 73f8344b-aadb-4eda-971a-b52bf9a56847, kind: Service { id: 0214b950-30f3-4a60-9344-b4f7d110e307 }, name: Name("nexus-0214b950-30f3-4a60-9344-b4f7d110e307"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 172, 92])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 470a6d71-26ee-4d59-b30a-7a9e78dfa903, kind: Service { id: 709da000-e0da-4b29-8160-3112fd080927 }, name: Name("external-dns-709da000-e0da-4b29-8160-3112fd080927"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 160, 53])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8dee8d46-edc9-4f8a-9453-2146d512c47b, kind: Service { id: a03854a6-7a78-402b-93b4-65c6d36bf236 }, name: Name("external-dns-a03854a6-7a78-402b-93b4-65c6d36bf236"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 200])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
8117	2024-12-06T21:24:54.812Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8118	2024-12-06T21:24:54.812Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8119	2024-12-06T21:24:54.813Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8120	2024-12-06T21:24:54.815Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
8121	2024-12-06T21:24:54.815Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8122	2024-12-06T21:24:54.815Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8123	2024-12-06T21:24:54.817Z	DEBG	nexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
8124	2024-12-06T21:24:54.817Z	DEBG	nexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 470a6d71-26ee-4d59-b30a-7a9e78dfa903, kind: Service { id: 709da000-e0da-4b29-8160-3112fd080927 }, name: Name("external-dns-709da000-e0da-4b29-8160-3112fd080927"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 160, 53])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8dee8d46-edc9-4f8a-9453-2146d512c47b, kind: Service { id: a03854a6-7a78-402b-93b4-65c6d36bf236 }, name: Name("external-dns-a03854a6-7a78-402b-93b4-65c6d36bf236"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 200])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 112d04a8-d7b2-41ba-a8a9-b1b05b82789a, kind: Service { id: e8725c8c-13ab-4916-bf16-bb7f832014d5 }, name: Name("nexus-e8725c8c-13ab-4916-bf16-bb7f832014d5"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 245, 172])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 3b455a5c-72a6-4c48-bc31-0f3337db4efa, kind: Service { id: a211d4f7-7350-4466-97ca-36b58597b187 }, name: Name("nexus-a211d4f7-7350-4466-97ca-36b58597b187"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 181, 41])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 73f8344b-aadb-4eda-971a-b52bf9a56847, kind: Service { id: 0214b950-30f3-4a60-9344-b4f7d110e307 }, name: Name("nexus-0214b950-30f3-4a60-9344-b4f7d110e307"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 172, 92])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
8125	2024-12-06T21:24:54.817Z	DEBG	nexus (service-firewall-rule-progator): resolved 2 rules for sleds background_task = service_firewall_rule_propagation
8126	2024-12-06T21:24:54.828Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8127	2024-12-06T21:24:54.828Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8128	2024-12-06T21:24:54.830Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8129	2024-12-06T21:24:54.832Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
8130	2024-12-06T21:24:54.832Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8131	2024-12-06T21:24:54.833Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8132	2024-12-06T21:24:54.834Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8133	2024-12-06T21:24:54.836Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
8134	2024-12-06T21:24:54.836Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8135	2024-12-06T21:24:54.836Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8136	2024-12-06T21:24:54.864Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
8137	2024-12-06T21:24:54.865Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0a3762c-de00-4f78-b446-38a221420564", "content-length": "1", "date": "Fri, 06 Dec 2024 21:24:54 GMT"} })
8138	2024-12-06T21:24:54.866Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
8139	2024-12-06T21:24:54.867Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
8140	2024-12-06T21:24:54.869Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 461.993627ms iteration = 21 status = null
8141	2024-12-06T21:24:54.875Z	DEBG	nexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: cbc5c1c6-5668-455a-85ef-94f4aebe0f80, time_created: 2024-12-06T21:15:03.870849Z, time_modified: 2024-12-06T21:15:03.870849Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 2d75e4a3-6363-4fd1-935e-989d31fd7d71, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(51474595840)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)) }]
8142	2024-12-06T21:24:54.875Z	DEBG	nexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
8143	2024-12-06T21:24:54.875Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8144	2024-12-06T21:24:54.876Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8145	2024-12-06T21:24:54.878Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8146	2024-12-06T21:24:54.878Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8147	2024-12-06T21:24:54.880Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8148	2024-12-06T21:24:54.883Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: cbc5c1c6-5668-455a-85ef-94f4aebe0f80, lookup_type: ById(cbc5c1c6-5668-455a-85ef-94f4aebe0f80) } result = Ok(())
8149	2024-12-06T21:24:54.935Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
8150	2024-12-06T21:24:54.936Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "cd4ff362-5c8c-4a15-873a-61c92693209c", "date": "Fri, 06 Dec 2024 21:24:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
8151	2024-12-06T21:24:54.936Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
8152	2024-12-06T21:24:54.937Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "9762fe1b-3cc4-45f3-b198-97d63c6b70a6", "date": "Fri, 06 Dec 2024 21:24:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
8153	2024-12-06T21:24:54.937Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
8154	2024-12-06T21:24:54.940Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "9f7128fd-8efe-4eb6-a6f1-729a453c789a", "date": "Fri, 06 Dec 2024 21:24:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
8155	2024-12-06T21:24:54.940Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
8156	2024-12-06T21:24:54.943Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
8157	2024-12-06T21:24:54.943Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 535.974721ms iteration = 13 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
8158	2024-12-06T21:24:54.954Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
8159	2024-12-06T21:24:54.954Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "ca0ce4cc-9fdd-4976-9577-497be9a155be", "date": "Fri, 06 Dec 2024 21:24:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
8160	2024-12-06T21:24:54.955Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
8161	2024-12-06T21:24:54.957Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "10402edb-3e20-491a-be94-96afd4367d39", "date": "Fri, 06 Dec 2024 21:24:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
8162	2024-12-06T21:24:54.957Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
8163	2024-12-06T21:24:54.960Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
8164	2024-12-06T21:24:54.960Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 553.065532ms iteration = 13 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
8165	2024-12-06T21:24:54.980Z	DEBG	nexus (BackgroundTasks): begin collection background_task = inventory_collection
8166	2024-12-06T21:24:54.980Z	DEBG	nexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
8167	2024-12-06T21:24:54.980Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
8168	2024-12-06T21:24:54.981Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "23739127-1c6b-436e-8210-cdc91b633c02", "content-length": "476", "date": "Fri, 06 Dec 2024 21:24:54 GMT"} })
8169	2024-12-06T21:24:54.981Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
8170	2024-12-06T21:24:54.981Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f7ea1203-1810-43b8-9fb3-5ebc019916e8", "content-length": "734", "date": "Fri, 06 Dec 2024 21:24:54 GMT"} })
8171	2024-12-06T21:24:54.981Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
8172	2024-12-06T21:24:54.981Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d2ec00b-0ce8-4655-b7bb-7893ff70ce7e", "content-length": "111", "date": "Fri, 06 Dec 2024 21:24:54 GMT"} })
8173	2024-12-06T21:24:54.981Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
8174	2024-12-06T21:24:54.982Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "349c04d9-87ca-441e-bb7a-1a5aefd600f2", "content-length": "111", "date": "Fri, 06 Dec 2024 21:24:54 GMT"} })
8175	2024-12-06T21:24:54.982Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
8176	2024-12-06T21:24:54.982Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d79d77c3-6ff4-4d44-93cd-ad45414107bf", "content-length": "167", "date": "Fri, 06 Dec 2024 21:24:54 GMT"} })
8177	2024-12-06T21:24:54.982Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
8178	2024-12-06T21:24:54.983Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9b01d34d-0cd4-4571-948b-d29156e0c9eb", "content-length": "167", "date": "Fri, 06 Dec 2024 21:24:54 GMT"} })
8179	2024-12-06T21:24:54.983Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
8180	2024-12-06T21:24:54.983Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "27e007e5-d04f-45c9-94ca-60e8b03067ac", "content-length": "175", "date": "Fri, 06 Dec 2024 21:24:54 GMT"} })
8181	2024-12-06T21:24:54.983Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
8182	2024-12-06T21:24:54.983Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cdb9b415-212e-4360-95a7-eb9599302f6f", "content-length": "175", "date": "Fri, 06 Dec 2024 21:24:54 GMT"} })
8183	2024-12-06T21:24:54.983Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
8184	2024-12-06T21:24:54.983Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "eb4fb229-69a6-41d2-a749-1b5ce8716bc0", "content-length": "702", "date": "Fri, 06 Dec 2024 21:24:54 GMT"} })
8185	2024-12-06T21:24:54.983Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
8186	2024-12-06T21:24:54.984Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "80588c6e-b59b-42e2-9aba-a2319dfadf25", "content-length": "727", "date": "Fri, 06 Dec 2024 21:24:54 GMT"} })
8187	2024-12-06T21:24:54.984Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
8188	2024-12-06T21:24:54.984Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a76f728c-df74-4866-a5c1-a3854475d10a", "content-length": "729", "date": "Fri, 06 Dec 2024 21:24:54 GMT"} })
8189	2024-12-06T21:24:54.984Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
8190	2024-12-06T21:24:54.984Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a043131-22a6-4482-a996-a4e58994817d", "content-length": "728", "date": "Fri, 06 Dec 2024 21:24:54 GMT"} })
8191	2024-12-06T21:24:54.984Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
8192	2024-12-06T21:24:54.984Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6fdcb1d-6f1b-4c89-806c-31eafd5abd6e", "content-length": "722", "date": "Fri, 06 Dec 2024 21:24:54 GMT"} })
8193	2024-12-06T21:24:54.984Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
8194	2024-12-06T21:24:54.985Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "75461a6a-9d06-4396-9156-4bf36911d3c5", "content-length": "109", "date": "Fri, 06 Dec 2024 21:24:54 GMT"} })
8195	2024-12-06T21:24:54.985Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
8196	2024-12-06T21:24:54.986Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2120aed4-54af-4d5d-9494-1f5c9c4fc9c2", "content-length": "109", "date": "Fri, 06 Dec 2024 21:24:54 GMT"} })
8197	2024-12-06T21:24:54.986Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
8198	2024-12-06T21:24:54.986Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "abdf0e31-966f-4754-b6ba-10353bf3dd4c", "content-length": "169", "date": "Fri, 06 Dec 2024 21:24:54 GMT"} })
8199	2024-12-06T21:24:54.986Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
8200	2024-12-06T21:24:54.988Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3cfc2d34-d8b6-4922-9ab5-36229a2e8b73", "content-length": "169", "date": "Fri, 06 Dec 2024 21:24:54 GMT"} })
8201	2024-12-06T21:24:54.988Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
8202	2024-12-06T21:24:54.988Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "43d415cf-fc20-4617-85d6-80f7e9740506", "content-length": "178", "date": "Fri, 06 Dec 2024 21:24:54 GMT"} })
8203	2024-12-06T21:24:54.988Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
8204	2024-12-06T21:24:54.988Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "26be97af-d29e-46ed-84c8-ab36bd68be46", "content-length": "178", "date": "Fri, 06 Dec 2024 21:24:54 GMT"} })
8205	2024-12-06T21:24:54.988Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
8206	2024-12-06T21:24:54.988Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a198d611-a7e6-4dd9-81f3-56eb92eca862", "content-length": "702", "date": "Fri, 06 Dec 2024 21:24:54 GMT"} })
8207	2024-12-06T21:24:54.988Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
8208	2024-12-06T21:24:54.989Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "73a2c51f-bcb9-4a7d-a853-19753ad73c6a", "content-length": "727", "date": "Fri, 06 Dec 2024 21:24:54 GMT"} })
8209	2024-12-06T21:24:54.989Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
8210	2024-12-06T21:24:54.989Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f01713b1-1888-44cd-8cf4-63a95b306391", "content-length": "729", "date": "Fri, 06 Dec 2024 21:24:54 GMT"} })
8211	2024-12-06T21:24:54.989Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
8212	2024-12-06T21:24:54.989Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "36c3c596-9713-45c1-92a8-4c77af2b56b7", "content-length": "728", "date": "Fri, 06 Dec 2024 21:24:54 GMT"} })
8213	2024-12-06T21:24:54.989Z	DEBG	nexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
8214	2024-12-06T21:24:54.989Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
8215	2024-12-06T21:24:54.991Z	DEBG	nexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8216	2024-12-06T21:24:54.992Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
8217	2024-12-06T21:24:54.992Z	DEBG	nexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
8218	2024-12-06T21:24:54.992Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
8219	2024-12-06T21:24:55.005Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 597.532361ms iteration = 21 status = {}
8220	2024-12-06T21:24:55.038Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
8221	2024-12-06T21:24:55.039Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "b39b64bb-7970-41de-8752-d7702a5c392f", "content-length": "691", "date": "Fri, 06 Dec 2024 21:24:55 GMT"} })
8222	2024-12-06T21:24:55.039Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
8223	2024-12-06T21:24:55.039Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
8224	2024-12-06T21:24:55.039Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
8225	2024-12-06T21:24:55.040Z	DEBG	nexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8226	2024-12-06T21:24:55.040Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 633.190266ms iteration = 21 status = {}
8227	2024-12-06T21:24:55.042Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
8228	2024-12-06T21:24:55.042Z	DEBG	nexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
8229	2024-12-06T21:24:55.042Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
8230	2024-12-06T21:24:55.089Z	DEBG	nexus (service-firewall-rule-progator): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
8231	2024-12-06T21:24:55.090Z	DEBG	nexus (service-firewall-rule-progator): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "2130257c-6d54-494a-9299-8c415f91e1d5", "date": "Fri, 06 Dec 2024 21:24:55 GMT"} })
8232	2024-12-06T21:24:55.091Z	INFO	nexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:496
8233	2024-12-06T21:24:55.091Z	DEBG	nexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 683.977928ms
8234	2024-12-06T21:24:55.091Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 683.987479ms iteration = 3 status = {}
8235	2024-12-06T21:24:55.153Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8236	2024-12-06T21:24:55.154Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8237	2024-12-06T21:24:55.156Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 3d56b7bf-53ed-4147-b45b-9cef154d7338, rack_id: 2d75e4a3-6363-4fd1-935e-989d31fd7d71, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(02859c9b-92d7-4b19-b425-8f266763be54) } rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
8238	2024-12-06T21:24:55.156Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8239	2024-12-06T21:24:55.156Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8240	2024-12-06T21:24:55.161Z	DEBG	nexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
8241	2024-12-06T21:24:55.161Z	DEBG	nexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
8242	2024-12-06T21:24:55.168Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
8243	2024-12-06T21:24:55.168Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "90214f6e-a8cb-4a15-bec7-731433185266", "content-length": "134", "date": "Fri, 06 Dec 2024 21:24:54 GMT"} })
8244	2024-12-06T21:24:55.168Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
8245	2024-12-06T21:24:55.168Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 02859c9b-92d7-4b19-b425-8f266763be54
8246	2024-12-06T21:24:55.168Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
8247	2024-12-06T21:24:55.168Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
8248	2024-12-06T21:24:55.168Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8249	2024-12-06T21:24:55.168Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8250	2024-12-06T21:24:55.168Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8251	2024-12-06T21:24:55.168Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
8252	2024-12-06T21:24:55.168Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8253	2024-12-06T21:24:55.168Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
8254	2024-12-06T21:24:55.169Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
8255	2024-12-06T21:24:55.169Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8256	2024-12-06T21:24:55.169Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8257	2024-12-06T21:24:55.171Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
8258	2024-12-06T21:24:55.171Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "4d1101be-fdfb-401a-9246-a4fc411e2531", "content-length": "2", "date": "Fri, 06 Dec 2024 21:24:54 GMT"} })
8259	2024-12-06T21:24:55.171Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
8260	2024-12-06T21:24:55.172Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "18fc6955-9822-44a7-9dfd-1081d5643a6e", "content-length": "52", "date": "Fri, 06 Dec 2024 21:24:54 GMT"} })
8261	2024-12-06T21:24:55.172Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
8262	2024-12-06T21:24:55.172Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
8263	2024-12-06T21:24:55.293Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "19d00488-0dcc-44fd-a0e9-181a1ceee207", "date": "Fri, 06 Dec 2024 21:24:55 GMT"} })
8264	2024-12-06T21:24:55.293Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch = Switch0
8265	2024-12-06T21:24:55.293Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
8266	2024-12-06T21:24:55.298Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "4db0ab7a-890c-4193-bc1f-adcdb7e44bc0", "content-length": "682", "date": "Fri, 06 Dec 2024 21:24:55 GMT"} })
8267	2024-12-06T21:24:55.298Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8268	2024-12-06T21:24:55.298Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8269	2024-12-06T21:24:55.302Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8270	2024-12-06T21:24:55.302Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8271	2024-12-06T21:24:55.305Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8272	2024-12-06T21:24:55.306Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8273	2024-12-06T21:24:55.308Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8274	2024-12-06T21:24:55.308Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8275	2024-12-06T21:24:55.315Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 908.152456ms iteration = 21 status = {}
8276	2024-12-06T21:24:55.496Z	DEBG	nexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e82e09f-f91d-4c53-80ee-2a62bd9a2dc3", "content-length": "30798", "date": "Fri, 06 Dec 2024 21:24:55 GMT"} })
8277	2024-12-06T21:24:55.513Z	DEBG	nexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
8278	2024-12-06T21:24:55.513Z	DEBG	nexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
8279	2024-12-06T21:24:55.513Z	DEBG	nexus (BackgroundTasks): finished collection background_task = inventory_collection
8280	2024-12-06T21:24:55.517Z	DEBG	nexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
8281	2024-12-06T21:24:55.517Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
8282	2024-12-06T21:24:55.519Z	DEBG	nexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8283	2024-12-06T21:24:55.521Z	DEBG	nexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
8284	2024-12-06T21:24:55.523Z	DEBG	nexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
8285	2024-12-06T21:24:55.523Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
8286	2024-12-06T21:24:55.716Z	INFO	nexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = bd705c2e-fd12-40af-bb9e-ac52a250a6d9 file = nexus/db-queries/src/db/datastore/inventory.rs:1010
8287	2024-12-06T21:24:55.716Z	DEBG	nexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = bd705c2e-fd12-40af-bb9e-ac52a250a6d9 time_started = 2024-12-06 21:24:54.980526 UTC
8288	2024-12-06T21:24:55.716Z	DEBG	nexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 1.30963234s iteration = 2 status = {"collection_id":"bd705c2e-fd12-40af-bb9e-ac52a250a6d9","time_done":"2024-12-06 21:24:55.516145 UTC","time_started":"2024-12-06 21:24:54.980526 UTC"}
8289	2024-12-06T21:24:55.716Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 23 reason = Dependency
8290	2024-12-06T21:24:55.716Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
8291	2024-12-06T21:24:55.717Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
8292	2024-12-06T21:24:55.718Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8293	2024-12-06T21:24:55.720Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
8294	2024-12-06T21:24:55.720Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
8295	2024-12-06T21:24:55.720Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
8296	2024-12-06T21:24:55.722Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
8297	2024-12-06T21:24:55.723Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
8298	2024-12-06T21:24:55.724Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8299	2024-12-06T21:24:55.726Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
8300	2024-12-06T21:24:55.726Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
8301	2024-12-06T21:24:55.726Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
8302	2024-12-06T21:24:55.749Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 32.413882ms iteration = 23 status = {"physical_disks_added":0}
8303	2024-12-06T21:24:56.542Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 272 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 6544aba3-057e-4ac5-8f65-5143e97db62c response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
8304	2024-12-06T21:25:04.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 62 reason = Timeout
8305	2024-12-06T21:25:04.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 62 reason = Timeout
8306	2024-12-06T21:25:04.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8307	2024-12-06T21:25:04.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8308	2024-12-06T21:25:04.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8309	2024-12-06T21:25:04.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8310	2024-12-06T21:25:04.410Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8311	2024-12-06T21:25:04.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8312	2024-12-06T21:25:04.412Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8313	2024-12-06T21:25:04.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8314	2024-12-06T21:25:04.414Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8315	2024-12-06T21:25:04.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8316	2024-12-06T21:25:04.416Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8317	2024-12-06T21:25:04.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
8318	2024-12-06T21:25:04.419Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8319	2024-12-06T21:25:04.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8320	2024-12-06T21:25:04.431Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
8321	2024-12-06T21:25:04.431Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.37924ms iteration = 62 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
8322	2024-12-06T21:25:04.435Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8323	2024-12-06T21:25:04.436Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8324	2024-12-06T21:25:04.464Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 56.859993ms iteration = 62 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
8325	2024-12-06T21:25:06.535Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54925 req_id = 88a42159-2aa8-4d8a-97e6-aa30684be260 roles = RoleSet { roles: {} } uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
8326	2024-12-06T21:25:06.536Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54925 req_id = 88a42159-2aa8-4d8a-97e6-aa30684be260 resource = Database result = Ok(()) uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
8327	2024-12-06T21:25:06.538Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2966 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54925 req_id = 88a42159-2aa8-4d8a-97e6-aa30684be260 response_code = 200 uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
8328	2024-12-06T21:25:06.538Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54925 req_id = 4531c2d8-92e9-4d6b-a8f1-b2d978111abd roles = RoleSet { roles: {} } uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
8329	2024-12-06T21:25:06.539Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54925 req_id = 4531c2d8-92e9-4d6b-a8f1-b2d978111abd resource = Database result = Ok(()) uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
8330	2024-12-06T21:25:06.540Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2184 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54925 req_id = 4531c2d8-92e9-4d6b-a8f1-b2d978111abd response_code = 200 uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
8331	2024-12-06T21:25:06.541Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 259 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 9014015d-2b15-4d98-96c5-2ed20285758d response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
8332	2024-12-06T21:25:14.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 63 reason = Timeout
8333	2024-12-06T21:25:14.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 63 reason = Timeout
8334	2024-12-06T21:25:14.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8335	2024-12-06T21:25:14.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8336	2024-12-06T21:25:14.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8337	2024-12-06T21:25:14.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8338	2024-12-06T21:25:14.409Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8339	2024-12-06T21:25:14.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8340	2024-12-06T21:25:14.411Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8341	2024-12-06T21:25:14.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8342	2024-12-06T21:25:14.414Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8343	2024-12-06T21:25:14.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8344	2024-12-06T21:25:14.416Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8345	2024-12-06T21:25:14.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
8346	2024-12-06T21:25:14.418Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8347	2024-12-06T21:25:14.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8348	2024-12-06T21:25:14.431Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
8349	2024-12-06T21:25:14.431Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.897932ms iteration = 63 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
8350	2024-12-06T21:25:14.437Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8351	2024-12-06T21:25:14.437Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8352	2024-12-06T21:25:14.465Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 58.351987ms iteration = 63 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
8353	2024-12-06T21:25:16.542Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 298 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 267abc1c-1697-4c25-bb75-68eeac240b41 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
8354	2024-12-06T21:25:21.535Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54925 req_id = 7ce5dac4-7978-4b9a-8bb7-7744316b542c roles = RoleSet { roles: {} } uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
8355	2024-12-06T21:25:21.535Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54925 req_id = 7ce5dac4-7978-4b9a-8bb7-7744316b542c resource = Database result = Ok(()) uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
8356	2024-12-06T21:25:21.538Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3222 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54925 req_id = 7ce5dac4-7978-4b9a-8bb7-7744316b542c response_code = 200 uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
8357	2024-12-06T21:25:21.538Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54925 req_id = 7dfce1fe-38ce-4620-aab8-ceea1f95afc2 roles = RoleSet { roles: {} } uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
8358	2024-12-06T21:25:21.539Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54925 req_id = 7dfce1fe-38ce-4620-aab8-ceea1f95afc2 resource = Database result = Ok(()) uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
8359	2024-12-06T21:25:21.541Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2214 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54925 req_id = 7dfce1fe-38ce-4620-aab8-ceea1f95afc2 response_code = 200 uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
8360	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 22 reason = Timeout
8361	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8362	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 22 reason = Timeout
8363	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
8364	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 22 reason = Timeout
8365	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 22 reason = Timeout
8366	2024-12-06T21:25:24.407Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
8367	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8368	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8369	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 22 reason = Timeout
8370	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 22 reason = Timeout
8371	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
8372	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 64 reason = Timeout
8373	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8374	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8375	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 64 reason = Timeout
8376	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 22 reason = Timeout
8377	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
8378	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8379	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 22 reason = Timeout
8380	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
8381	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 22 reason = Timeout
8382	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 22 reason = Timeout
8383	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
8384	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 22 reason = Timeout
8385	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 22 reason = Timeout
8386	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
8387	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 24 reason = Timeout
8388	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
8389	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
8390	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 22 reason = Timeout
8391	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
8392	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 22 reason = Timeout
8393	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
8394	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
8395	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8396	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8397	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8398	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
8399	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
8400	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
8401	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
8402	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
8403	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8404	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
8405	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
8406	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
8407	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8408	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
8409	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
8410	2024-12-06T21:25:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8411	2024-12-06T21:25:24.409Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8412	2024-12-06T21:25:24.410Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8413	2024-12-06T21:25:24.410Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.353724ms iteration = 22 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
8414	2024-12-06T21:25:24.410Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 3.500667ms iteration = 22 status = {"errors":[],"garbage_collect_requested":[]}
8415	2024-12-06T21:25:24.411Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1439, tv_nsec: 909002298 } })
8416	2024-12-06T21:25:24.411Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
8417	2024-12-06T21:25:24.411Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
8418	2024-12-06T21:25:24.411Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8419	2024-12-06T21:25:24.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8420	2024-12-06T21:25:24.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
8421	2024-12-06T21:25:24.412Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8422	2024-12-06T21:25:24.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8423	2024-12-06T21:25:24.414Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8424	2024-12-06T21:25:24.414Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8425	2024-12-06T21:25:24.416Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1439, tv_nsec: 913672965 } })
8426	2024-12-06T21:25:24.416Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
8427	2024-12-06T21:25:24.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
8428	2024-12-06T21:25:24.417Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
8429	2024-12-06T21:25:24.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
8430	2024-12-06T21:25:24.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
8431	2024-12-06T21:25:24.417Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8432	2024-12-06T21:25:24.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8433	2024-12-06T21:25:24.418Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8434	2024-12-06T21:25:24.419Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8435	2024-12-06T21:25:24.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8436	2024-12-06T21:25:24.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8437	2024-12-06T21:25:24.421Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8438	2024-12-06T21:25:24.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8439	2024-12-06T21:25:24.424Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1439, tv_nsec: 922091333 } })
8440	2024-12-06T21:25:24.424Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
8441	2024-12-06T21:25:24.425Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8442	2024-12-06T21:25:24.425Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8443	2024-12-06T21:25:24.425Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8444	2024-12-06T21:25:24.425Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8445	2024-12-06T21:25:24.433Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8446	2024-12-06T21:25:24.433Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8447	2024-12-06T21:25:24.449Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
8448	2024-12-06T21:25:24.449Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
8449	2024-12-06T21:25:24.451Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
8450	2024-12-06T21:25:24.451Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
8451	2024-12-06T21:25:24.455Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8452	2024-12-06T21:25:24.455Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8453	2024-12-06T21:25:24.456Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8454	2024-12-06T21:25:24.457Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
8455	2024-12-06T21:25:24.457Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
8456	2024-12-06T21:25:24.457Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8457	2024-12-06T21:25:24.457Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
8458	2024-12-06T21:25:24.457Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
8459	2024-12-06T21:25:24.457Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
8460	2024-12-06T21:25:24.457Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
8461	2024-12-06T21:25:24.458Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
8462	2024-12-06T21:25:24.458Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
8463	2024-12-06T21:25:24.458Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
8464	2024-12-06T21:25:24.459Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8465	2024-12-06T21:25:24.459Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8466	2024-12-06T21:25:24.459Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
8467	2024-12-06T21:25:24.459Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8468	2024-12-06T21:25:24.460Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8469	2024-12-06T21:25:24.460Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
8470	2024-12-06T21:25:24.460Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
8471	2024-12-06T21:25:24.460Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
8472	2024-12-06T21:25:24.460Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
8473	2024-12-06T21:25:24.460Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 53.448466ms iteration = 22 status = {"errors":[],"records_set_to_done":[]}
8474	2024-12-06T21:25:24.460Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
8475	2024-12-06T21:25:24.461Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
8476	2024-12-06T21:25:24.461Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8477	2024-12-06T21:25:24.462Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8478	2024-12-06T21:25:24.463Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 56.689179ms iteration = 64 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
8479	2024-12-06T21:25:24.464Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8480	2024-12-06T21:25:24.466Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
8481	2024-12-06T21:25:24.466Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8482	2024-12-06T21:25:24.466Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8483	2024-12-06T21:25:24.471Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
8484	2024-12-06T21:25:24.471Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 64.279864ms iteration = 22 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
8485	2024-12-06T21:25:24.485Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8486	2024-12-06T21:25:24.487Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
8487	2024-12-06T21:25:24.487Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8488	2024-12-06T21:25:24.487Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8489	2024-12-06T21:25:24.493Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 86.52188ms iteration = 22 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
8490	2024-12-06T21:25:24.495Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
8491	2024-12-06T21:25:24.495Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
8492	2024-12-06T21:25:24.496Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8493	2024-12-06T21:25:24.496Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 89.673391ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
8494	2024-12-06T21:25:24.497Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8495	2024-12-06T21:25:24.498Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
8496	2024-12-06T21:25:24.498Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
8497	2024-12-06T21:25:24.498Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
8498	2024-12-06T21:25:24.500Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cbc5c1c6-5668-455a-85ef-94f4aebe0f80, lookup_type: ById(cbc5c1c6-5668-455a-85ef-94f4aebe0f80) } result = Ok(())
8499	2024-12-06T21:25:24.500Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8500	2024-12-06T21:25:24.500Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8501	2024-12-06T21:25:24.502Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
8502	2024-12-06T21:25:24.502Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
8503	2024-12-06T21:25:24.502Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 95.442315ms iteration = 22 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
8504	2024-12-06T21:25:24.503Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
8505	2024-12-06T21:25:24.503Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
8506	2024-12-06T21:25:24.504Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "91ca725f-1b84-47d4-b0ec-6ef8a4979759", "content-length": "26", "date": "Fri, 06 Dec 2024 21:25:24 GMT"} })
8507	2024-12-06T21:25:24.504Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
8508	2024-12-06T21:25:24.504Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
8509	2024-12-06T21:25:24.504Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
8510	2024-12-06T21:25:24.505Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "24786ca4-d731-4615-8f43-bb844b184d28", "content-length": "26", "date": "Fri, 06 Dec 2024 21:25:24 GMT"} })
8511	2024-12-06T21:25:24.505Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
8512	2024-12-06T21:25:24.506Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
8513	2024-12-06T21:25:24.507Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
8514	2024-12-06T21:25:24.525Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 zone_address = fd00:1122:3344:101::2
8515	2024-12-06T21:25:24.525Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
8516	2024-12-06T21:25:24.525Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1f6b480-deef-49d2-8298-f783235dde41", "content-length": "26", "date": "Fri, 06 Dec 2024 21:25:24 GMT"} })
8517	2024-12-06T21:25:24.525Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
8518	2024-12-06T21:25:24.526Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
8519	2024-12-06T21:25:24.547Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
8520	2024-12-06T21:25:24.547Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
8521	2024-12-06T21:25:24.547Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "1fa92933-1a49-4935-8676-b6cff05756f2", "content-length": "372", "date": "Fri, 06 Dec 2024 21:25:23 GMT"} })
8522	2024-12-06T21:25:24.547Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8523	2024-12-06T21:25:24.547Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8524	2024-12-06T21:25:24.549Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8525	2024-12-06T21:25:24.551Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 144.271817ms iteration = 22 status = 0
8526	2024-12-06T21:25:24.552Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
8527	2024-12-06T21:25:24.552Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8528	2024-12-06T21:25:24.552Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8529	2024-12-06T21:25:24.559Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
8530	2024-12-06T21:25:24.559Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 152.277507ms iteration = 64 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
8531	2024-12-06T21:25:24.578Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 170.783421ms iteration = 22 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
8532	2024-12-06T21:25:24.588Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 181.206382ms iteration = 22 status = {}
8533	2024-12-06T21:25:24.591Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
8534	2024-12-06T21:25:24.591Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
8535	2024-12-06T21:25:24.591Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "86daf37d-82dd-455e-8936-1c9854821fc8", "date": "Fri, 06 Dec 2024 21:25:23 GMT"} })
8536	2024-12-06T21:25:24.591Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8537	2024-12-06T21:25:24.591Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8538	2024-12-06T21:25:24.594Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8539	2024-12-06T21:25:24.594Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8540	2024-12-06T21:25:24.596Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
8541	2024-12-06T21:25:24.596Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "51fc6cb5-e09b-4cb5-ab9b-524d87318e0f", "content-length": "1", "date": "Fri, 06 Dec 2024 21:25:24 GMT"} })
8542	2024-12-06T21:25:24.597Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8543	2024-12-06T21:25:24.597Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8544	2024-12-06T21:25:24.597Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
8545	2024-12-06T21:25:24.597Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
8546	2024-12-06T21:25:24.600Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 192.930015ms iteration = 22 status = null
8547	2024-12-06T21:25:24.600Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
8548	2024-12-06T21:25:24.600Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
8549	2024-12-06T21:25:24.601Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "aabca7e1-9aa2-4ca2-8c03-6847c5333455", "date": "Fri, 06 Dec 2024 21:25:23 GMT"} })
8550	2024-12-06T21:25:24.602Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 195.130701ms iteration = 22 status = {}
8551	2024-12-06T21:25:24.663Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 255.879967ms iteration = 24 status = {"physical_disks_added":0}
8552	2024-12-06T21:25:24.690Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
8553	2024-12-06T21:25:24.691Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "20500e3a-327e-437d-8c40-6eb5ec33a714", "content-length": "691", "date": "Fri, 06 Dec 2024 21:25:23 GMT"} })
8554	2024-12-06T21:25:24.691Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
8555	2024-12-06T21:25:24.691Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
8556	2024-12-06T21:25:24.691Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
8557	2024-12-06T21:25:24.692Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 285.451192ms iteration = 22 status = {}
8558	2024-12-06T21:25:24.704Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8559	2024-12-06T21:25:24.705Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8560	2024-12-06T21:25:24.707Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 3d56b7bf-53ed-4147-b45b-9cef154d7338, rack_id: 2d75e4a3-6363-4fd1-935e-989d31fd7d71, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(02859c9b-92d7-4b19-b425-8f266763be54) } rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
8561	2024-12-06T21:25:24.707Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8562	2024-12-06T21:25:24.707Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8563	2024-12-06T21:25:24.718Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
8564	2024-12-06T21:25:24.718Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b197179-dece-4eb6-a5e8-549f52d14ea0", "content-length": "134", "date": "Fri, 06 Dec 2024 21:25:24 GMT"} })
8565	2024-12-06T21:25:24.718Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
8566	2024-12-06T21:25:24.718Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 02859c9b-92d7-4b19-b425-8f266763be54
8567	2024-12-06T21:25:24.718Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
8568	2024-12-06T21:25:24.719Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
8569	2024-12-06T21:25:24.719Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8570	2024-12-06T21:25:24.719Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8571	2024-12-06T21:25:24.719Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8572	2024-12-06T21:25:24.719Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
8573	2024-12-06T21:25:24.719Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8574	2024-12-06T21:25:24.719Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
8575	2024-12-06T21:25:24.719Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
8576	2024-12-06T21:25:24.719Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8577	2024-12-06T21:25:24.719Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8578	2024-12-06T21:25:24.721Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
8579	2024-12-06T21:25:24.721Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0b4cbdb-b956-42fa-8eac-a5a59f298178", "content-length": "2", "date": "Fri, 06 Dec 2024 21:25:24 GMT"} })
8580	2024-12-06T21:25:24.721Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
8581	2024-12-06T21:25:24.721Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "541d656b-6252-4375-a894-ec29a1f39a82", "content-length": "52", "date": "Fri, 06 Dec 2024 21:25:24 GMT"} })
8582	2024-12-06T21:25:24.721Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
8583	2024-12-06T21:25:24.721Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
8584	2024-12-06T21:25:24.822Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "1df507cc-5eb1-448e-ba86-8fd2cacad6a5", "date": "Fri, 06 Dec 2024 21:25:24 GMT"} })
8585	2024-12-06T21:25:24.822Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch = Switch0
8586	2024-12-06T21:25:24.822Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
8587	2024-12-06T21:25:24.822Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "ea85549e-eac8-48b6-86ec-ce86e66caad3", "content-length": "682", "date": "Fri, 06 Dec 2024 21:25:24 GMT"} })
8588	2024-12-06T21:25:24.822Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8589	2024-12-06T21:25:24.822Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8590	2024-12-06T21:25:24.826Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8591	2024-12-06T21:25:24.826Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8592	2024-12-06T21:25:24.829Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8593	2024-12-06T21:25:24.829Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8594	2024-12-06T21:25:24.831Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8595	2024-12-06T21:25:24.831Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8596	2024-12-06T21:25:24.839Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 431.811802ms iteration = 22 status = {}
8597	2024-12-06T21:25:26.542Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 488 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 2c1c7cf3-af82-49e4-8f6c-b72d9ea45e4f response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
8598	2024-12-06T21:25:34.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 65 reason = Timeout
8599	2024-12-06T21:25:34.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 65 reason = Timeout
8600	2024-12-06T21:25:34.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8601	2024-12-06T21:25:34.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8602	2024-12-06T21:25:34.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8603	2024-12-06T21:25:34.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8604	2024-12-06T21:25:34.409Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8605	2024-12-06T21:25:34.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8606	2024-12-06T21:25:34.409Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8607	2024-12-06T21:25:34.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8608	2024-12-06T21:25:34.412Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8609	2024-12-06T21:25:34.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8610	2024-12-06T21:25:34.414Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8611	2024-12-06T21:25:34.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8612	2024-12-06T21:25:34.444Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8613	2024-12-06T21:25:34.445Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 38.234437ms iteration = 65 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
8614	2024-12-06T21:25:34.447Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
8615	2024-12-06T21:25:34.447Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8616	2024-12-06T21:25:34.447Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8617	2024-12-06T21:25:34.459Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
8618	2024-12-06T21:25:34.459Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 52.265917ms iteration = 65 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
8619	2024-12-06T21:25:36.541Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 263 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 56d0ce12-45d4-4754-9800-f55bcb7a1902 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
8620	2024-12-06T21:25:44.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 66 reason = Timeout
8621	2024-12-06T21:25:44.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 66 reason = Timeout
8622	2024-12-06T21:25:44.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8623	2024-12-06T21:25:44.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8624	2024-12-06T21:25:44.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8625	2024-12-06T21:25:44.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8626	2024-12-06T21:25:44.409Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8627	2024-12-06T21:25:44.409Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8628	2024-12-06T21:25:44.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8629	2024-12-06T21:25:44.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8630	2024-12-06T21:25:44.412Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8631	2024-12-06T21:25:44.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8632	2024-12-06T21:25:44.414Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8633	2024-12-06T21:25:44.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8634	2024-12-06T21:25:44.416Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8635	2024-12-06T21:25:44.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
8636	2024-12-06T21:25:44.418Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8637	2024-12-06T21:25:44.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8638	2024-12-06T21:25:44.429Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
8639	2024-12-06T21:25:44.429Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.555124ms iteration = 66 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
8640	2024-12-06T21:25:44.437Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 29.90344ms iteration = 66 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
8641	2024-12-06T21:25:46.542Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 264 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 45c8dc3f-10e7-4f61-9606-a9a24fc09938 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
8642	2024-12-06T21:25:51.535Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54925 req_id = 20b8bff3-06fb-496d-9914-3d5bd8ec788d roles = RoleSet { roles: {} } uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
8643	2024-12-06T21:25:51.536Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54925 req_id = 20b8bff3-06fb-496d-9914-3d5bd8ec788d resource = Database result = Ok(()) uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
8644	2024-12-06T21:25:51.539Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3611 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54925 req_id = 20b8bff3-06fb-496d-9914-3d5bd8ec788d response_code = 200 uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
8645	2024-12-06T21:25:51.539Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54925 req_id = 77412891-b815-44fe-a116-5f7be3d52284 roles = RoleSet { roles: {} } uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
8646	2024-12-06T21:25:51.539Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54925 req_id = 77412891-b815-44fe-a116-5f7be3d52284 resource = Database result = Ok(()) uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
8647	2024-12-06T21:25:51.542Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2856 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54925 req_id = 77412891-b815-44fe-a116-5f7be3d52284 response_code = 200 uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
8648	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 14 reason = Timeout
8649	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 23 reason = Timeout
8650	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
8651	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 12 reason = Timeout
8652	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
8653	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 23 reason = Timeout
8654	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 23 reason = Timeout
8655	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
8656	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
8657	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8658	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 23 reason = Timeout
8659	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
8660	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 23 reason = Timeout
8661	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 67 reason = Timeout
8662	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
8663	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8664	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = external_endpoints iteration = 12 reason = Timeout
8665	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8666	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 23 reason = Timeout
8667	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
8668	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 12 reason = Timeout
8669	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 23 reason = Timeout
8670	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 67 reason = Timeout
8671	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 23 reason = Timeout
8672	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 13 reason = Timeout
8673	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
8674	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8675	2024-12-06T21:25:54.407Z	WARN	nexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0
8676	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 12 reason = Timeout
8677	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 17.26\u{b5}s iteration = 13 status = {"enabled":false,"target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0"}
8678	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
8679	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 23 reason = Timeout
8680	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
8681	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
8682	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 12 reason = Timeout
8683	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
8684	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8685	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 23 reason = Timeout
8686	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 12 reason = Timeout
8687	2024-12-06T21:25:54.407Z	INFO	nexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
8688	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8689	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
8690	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 23 reason = Timeout
8691	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
8692	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 12 reason = Timeout
8693	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 23 reason = Timeout
8694	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
8695	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8696	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 12 reason = Timeout
8697	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 23 reason = Timeout
8698	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
8699	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 23 reason = Timeout
8700	2024-12-06T21:25:54.407Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
8701	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8702	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 12 reason = Timeout
8703	2024-12-06T21:25:54.407Z	INFO	nexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-06 21:15:54.407362275 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
8704	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
8705	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 25 reason = Timeout
8706	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
8707	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 14 reason = Timeout
8708	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 12 reason = Timeout
8709	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
8710	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
8711	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
8712	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
8713	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8714	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
8715	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
8716	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
8717	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
8718	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8719	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8720	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8721	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
8722	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
8723	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
8724	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
8725	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
8726	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
8727	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
8728	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8729	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8730	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8731	2024-12-06T21:25:54.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
8732	2024-12-06T21:25:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8733	2024-12-06T21:25:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
8734	2024-12-06T21:25:54.408Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("709da000-e0da-4b29-8160-3112fd080927.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a03854a6-7a78-402b-93b4-65c6d36bf236.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a03854a6-7a78-402b-93b4-65c6d36bf236.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1469, tv_nsec: 905546246 } })
8735	2024-12-06T21:25:54.408Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4aff1ae3-2d71-40bf-bc46-79a7276da3c2.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4dccdb26-9cf9-40aa-9229-5d35905cf0e1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("79d475dc-44ec-44f4-9e51-ea8a753c39f3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("79d475dc-44ec-44f4-9e51-ea8a753c39f3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1469, tv_nsec: 905597737 } })
8736	2024-12-06T21:25:54.408Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
8737	2024-12-06T21:25:54.408Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.188249ms iteration = 12 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
8738	2024-12-06T21:25:54.408Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
8739	2024-12-06T21:25:54.408Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.427554ms iteration = 12 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
8740	2024-12-06T21:25:54.410Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.440796ms iteration = 23 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
8741	2024-12-06T21:25:54.411Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8742	2024-12-06T21:25:54.412Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8743	2024-12-06T21:25:54.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
8744	2024-12-06T21:25:54.413Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8745	2024-12-06T21:25:54.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8746	2024-12-06T21:25:54.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
8747	2024-12-06T21:25:54.414Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
8748	2024-12-06T21:25:54.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
8749	2024-12-06T21:25:54.416Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8750	2024-12-06T21:25:54.417Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1469, tv_nsec: 915114383 } })
8751	2024-12-06T21:25:54.417Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
8752	2024-12-06T21:25:54.419Z	INFO	nexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-06 21:15:54.407362275 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
8753	2024-12-06T21:25:54.419Z	DEBG	nexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 11.679182ms iteration = 12 status = {"expiration":"2024-12-06T21:15:54.407362275Z","pruned":[]}
8754	2024-12-06T21:25:54.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8755	2024-12-06T21:25:54.419Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8756	2024-12-06T21:25:54.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8757	2024-12-06T21:25:54.422Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
8758	2024-12-06T21:25:54.423Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
8759	2024-12-06T21:25:54.424Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1469, tv_nsec: 921731091 } })
8760	2024-12-06T21:25:54.424Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
8761	2024-12-06T21:25:54.424Z	DEBG	nexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 17.429226ms iteration = 12 status = {"errors":[],"found_port_ok":[]}
8762	2024-12-06T21:25:54.424Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8763	2024-12-06T21:25:54.424Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8764	2024-12-06T21:25:54.425Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8765	2024-12-06T21:25:54.425Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8766	2024-12-06T21:25:54.428Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1469, tv_nsec: 926252925 } })
8767	2024-12-06T21:25:54.428Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
8768	2024-12-06T21:25:54.428Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
8769	2024-12-06T21:25:54.429Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
8770	2024-12-06T21:25:54.429Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
8771	2024-12-06T21:25:54.429Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
8772	2024-12-06T21:25:54.429Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8773	2024-12-06T21:25:54.430Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8774	2024-12-06T21:25:54.438Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8775	2024-12-06T21:25:54.441Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
8776	2024-12-06T21:25:54.441Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8777	2024-12-06T21:25:54.441Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8778	2024-12-06T21:25:54.451Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8779	2024-12-06T21:25:54.452Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8780	2024-12-06T21:25:54.452Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
8781	2024-12-06T21:25:54.452Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
8782	2024-12-06T21:25:54.454Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8783	2024-12-06T21:25:54.457Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
8784	2024-12-06T21:25:54.457Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8785	2024-12-06T21:25:54.457Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8786	2024-12-06T21:25:54.459Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8787	2024-12-06T21:25:54.459Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 52.611453ms iteration = 23 status = {"errors":[],"garbage_collect_requested":[]}
8788	2024-12-06T21:25:54.459Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
8789	2024-12-06T21:25:54.460Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
8790	2024-12-06T21:25:54.460Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
8791	2024-12-06T21:25:54.460Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
8792	2024-12-06T21:25:54.460Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
8793	2024-12-06T21:25:54.460Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
8794	2024-12-06T21:25:54.461Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8795	2024-12-06T21:25:54.461Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8796	2024-12-06T21:25:54.461Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8797	2024-12-06T21:25:54.461Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 54.731338ms iteration = 23 status = {"errors":[],"records_set_to_done":[]}
8798	2024-12-06T21:25:54.462Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
8799	2024-12-06T21:25:54.462Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
8800	2024-12-06T21:25:54.462Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
8801	2024-12-06T21:25:54.463Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
8802	2024-12-06T21:25:54.463Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8803	2024-12-06T21:25:54.463Z	DEBG	nexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
8804	2024-12-06T21:25:54.463Z	INFO	nexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
8805	2024-12-06T21:25:54.463Z	DEBG	nexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 56.603319ms iteration = 12 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
8806	2024-12-06T21:25:54.463Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8807	2024-12-06T21:25:54.469Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
8808	2024-12-06T21:25:54.469Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8809	2024-12-06T21:25:54.470Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
8810	2024-12-06T21:25:54.470Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8811	2024-12-06T21:25:54.472Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8812	2024-12-06T21:25:54.472Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
8813	2024-12-06T21:25:54.472Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
8814	2024-12-06T21:25:54.474Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
8815	2024-12-06T21:25:54.474Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
8816	2024-12-06T21:25:54.474Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
8817	2024-12-06T21:25:54.474Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8818	2024-12-06T21:25:54.476Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
8819	2024-12-06T21:25:54.476Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8820	2024-12-06T21:25:54.476Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8821	2024-12-06T21:25:54.480Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
8822	2024-12-06T21:25:54.480Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
8823	2024-12-06T21:25:54.480Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
8824	2024-12-06T21:25:54.480Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
8825	2024-12-06T21:25:54.490Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
8826	2024-12-06T21:25:54.490Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 83.686212ms iteration = 67 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
8827	2024-12-06T21:25:54.497Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8828	2024-12-06T21:25:54.500Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
8829	2024-12-06T21:25:54.500Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
8830	2024-12-06T21:25:54.500Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
8831	2024-12-06T21:25:54.500Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
8832	2024-12-06T21:25:54.500Z	DEBG	nexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
8833	2024-12-06T21:25:54.500Z	INFO	nexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
8834	2024-12-06T21:25:54.500Z	DEBG	nexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 92.922055ms iteration = 12 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
8835	2024-12-06T21:25:54.500Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
8836	2024-12-06T21:25:54.500Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
8837	2024-12-06T21:25:54.507Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 100.105652ms iteration = 67 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
8838	2024-12-06T21:25:54.509Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8839	2024-12-06T21:25:54.509Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8840	2024-12-06T21:25:54.510Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8841	2024-12-06T21:25:54.510Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
8842	2024-12-06T21:25:54.511Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
8843	2024-12-06T21:25:54.512Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
8844	2024-12-06T21:25:54.512Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
8845	2024-12-06T21:25:54.512Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
8846	2024-12-06T21:25:54.512Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 105.098694ms iteration = 23 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
8847	2024-12-06T21:25:54.512Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8848	2024-12-06T21:25:54.512Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8849	2024-12-06T21:25:54.513Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8850	2024-12-06T21:25:54.515Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
8851	2024-12-06T21:25:54.515Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
8852	2024-12-06T21:25:54.515Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 108.367367ms iteration = 23 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
8853	2024-12-06T21:25:54.515Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
8854	2024-12-06T21:25:54.515Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8855	2024-12-06T21:25:54.515Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8856	2024-12-06T21:25:54.515Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 108.53589ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
8857	2024-12-06T21:25:54.516Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
8858	2024-12-06T21:25:54.516Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
8859	2024-12-06T21:25:54.517Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8860	2024-12-06T21:25:54.520Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
8861	2024-12-06T21:25:54.520Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8862	2024-12-06T21:25:54.520Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8863	2024-12-06T21:25:54.537Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
8864	2024-12-06T21:25:54.537Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
8865	2024-12-06T21:25:54.537Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
8866	2024-12-06T21:25:54.537Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8867	2024-12-06T21:25:54.537Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8868	2024-12-06T21:25:54.554Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8869	2024-12-06T21:25:54.554Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8870	2024-12-06T21:25:54.563Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
8871	2024-12-06T21:25:54.577Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 170.147661ms iteration = 25 status = {"physical_disks_added":0}
8872	2024-12-06T21:25:54.577Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
8873	2024-12-06T21:25:54.577Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
8874	2024-12-06T21:25:54.577Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "10c5f424-4fdb-4a6c-98ad-d6fc1e9a3410", "content-length": "26", "date": "Fri, 06 Dec 2024 21:25:54 GMT"} })
8875	2024-12-06T21:25:54.577Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
8876	2024-12-06T21:25:54.579Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
8877	2024-12-06T21:25:54.594Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
8878	2024-12-06T21:25:54.594Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
8879	2024-12-06T21:25:54.594Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "256b5112-8361-4214-bca3-595256461429", "content-length": "26", "date": "Fri, 06 Dec 2024 21:25:54 GMT"} })
8880	2024-12-06T21:25:54.594Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
8881	2024-12-06T21:25:54.595Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
8882	2024-12-06T21:25:54.618Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
8883	2024-12-06T21:25:54.744Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 zone_address = fd00:1122:3344:101::2
8884	2024-12-06T21:25:54.744Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
8885	2024-12-06T21:25:54.744Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3456db7d-bfe3-48cc-b1ff-85a02f5c7944", "content-length": "26", "date": "Fri, 06 Dec 2024 21:25:54 GMT"} })
8886	2024-12-06T21:25:54.744Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
8887	2024-12-06T21:25:54.746Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
8888	2024-12-06T21:25:54.752Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
8889	2024-12-06T21:25:54.752Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "2c4643bc-20af-44e9-ad40-377809f2bab4", "date": "Fri, 06 Dec 2024 21:25:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
8890	2024-12-06T21:25:54.752Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
8891	2024-12-06T21:25:54.753Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "4436cd91-fef9-4cd1-9bff-263ce8e6224a", "date": "Fri, 06 Dec 2024 21:25:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
8892	2024-12-06T21:25:54.753Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
8893	2024-12-06T21:25:54.755Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
8894	2024-12-06T21:25:54.755Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 347.814865ms iteration = 14 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
8895	2024-12-06T21:25:54.764Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = instance_watcher body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vmms/bac2d732-e506-44e4-ae8b-e6d67d6a7be7/state
8896	2024-12-06T21:25:54.764Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8897	2024-12-06T21:25:54.764Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = instance_watcher result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vmms/bac2d732-e506-44e4-ae8b-e6d67d6a7be7/state", status: 200, headers: {"content-type": "application/json", "x-request-id": "e34afa6e-7b5c-4de2-9cb2-6856cf2d2925", "content-length": "130", "date": "Fri, 06 Dec 2024 21:25:54 GMT"} })
8898	2024-12-06T21:25:54.765Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8899	2024-12-06T21:25:54.766Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8900	2024-12-06T21:25:54.766Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8901	2024-12-06T21:25:54.768Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8902	2024-12-06T21:25:54.768Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cbc5c1c6-5668-455a-85ef-94f4aebe0f80, lookup_type: ById(cbc5c1c6-5668-455a-85ef-94f4aebe0f80) } result = Ok(())
8903	2024-12-06T21:25:54.768Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8904	2024-12-06T21:25:54.769Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8905	2024-12-06T21:25:54.769Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
8906	2024-12-06T21:25:54.769Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8907	2024-12-06T21:25:54.769Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8908	2024-12-06T21:25:54.770Z	DEBG	nexus (BackgroundTasks): updating instance state background_task = instance_watcher instance_id = 205c5de4-679a-4f08-8613-9b6f8d3a48e3 sled_id = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 state = SledVmmState { vmm_state: VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2024-12-06T21:25:52.449420701Z }, migration_in: None, migration_out: None } vmm_id = bac2d732-e506-44e4-ae8b-e6d67d6a7be7
8909	2024-12-06T21:25:54.770Z	INFO	nexus (BackgroundTasks): received new VMM runtime state from sled agent background_task = instance_watcher file = nexus/src/app/instance.rs:2116 instance_id = 205c5de4-679a-4f08-8613-9b6f8d3a48e3 migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = bac2d732-e506-44e4-ae8b-e6d67d6a7be7 sled_id = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 vmm_id = bac2d732-e506-44e4-ae8b-e6d67d6a7be7 vmm_state = VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2024-12-06T21:25:52.449420701Z }
8910	2024-12-06T21:25:54.770Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher instance_id = 205c5de4-679a-4f08-8613-9b6f8d3a48e3 roles = RoleSet { roles: {} } sled_id = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 vmm_id = bac2d732-e506-44e4-ae8b-e6d67d6a7be7
8911	2024-12-06T21:25:54.770Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = 205c5de4-679a-4f08-8613-9b6f8d3a48e3 resource = Database result = Ok(()) sled_id = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 vmm_id = bac2d732-e506-44e4-ae8b-e6d67d6a7be7
8912	2024-12-06T21:25:54.770Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
8913	2024-12-06T21:25:54.770Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8914	2024-12-06T21:25:54.770Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8915	2024-12-06T21:25:54.770Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 363.676535ms iteration = 23 status = 0
8916	2024-12-06T21:25:54.800Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
8917	2024-12-06T21:25:54.800Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 93c3f865-bc00-42df-bef7-20e39342f84f dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
8918	2024-12-06T21:25:54.800Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
8919	2024-12-06T21:25:54.800Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
8920	2024-12-06T21:25:54.800Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 393.449523ms iteration = 23 status = {}
8921	2024-12-06T21:25:54.803Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
8922	2024-12-06T21:25:54.803Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 05c53cbc-f66c-491f-af8d-6f291fe21596 dns_zone_name = oxide.test found_so_far = 0 version = 2
8923	2024-12-06T21:25:54.803Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8924	2024-12-06T21:25:54.803Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8925	2024-12-06T21:25:54.806Z	INFO	nexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-06T21:14:50.566199Z, time_modified: 2024-12-06T21:14:50.566199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, db_silo: Silo { identity: SiloIdentity { id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-06T21:15:06.991365Z, time_modified: 2024-12-06T21:15:06.991365Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, db_silo: Silo { identity: SiloIdentity { id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-06T21:15:06.991365Z, time_modified: 2024-12-06T21:15:06.991365Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:94
8926	2024-12-06T21:25:54.806Z	DEBG	nexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 399.088186ms iteration = 12 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"683d02e1-6ec8-48bf-98d0-db6c94785f80","tls_certs":["d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1"]}},"default_endpoint":{"silo_id":"683d02e1-6ec8-48bf-98d0-db6c94785f80","tls_certs":["d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
8927	2024-12-06T21:25:54.806Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 399.438612ms iteration = 23 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
8928	2024-12-06T21:25:54.807Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
8929	2024-12-06T21:25:54.807Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "546e895a-e451-490f-8368-b5e332c1ae7d", "content-length": "1", "date": "Fri, 06 Dec 2024 21:25:53 GMT"} })
8930	2024-12-06T21:25:54.809Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
8931	2024-12-06T21:25:54.810Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
8932	2024-12-06T21:25:54.813Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 406.550898ms iteration = 23 status = null
8933	2024-12-06T21:25:54.819Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
8934	2024-12-06T21:25:54.829Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 1 total_failed = 0 total_incomplete = 0 total_instances = 1 update_sagas_queued = 0
8935	2024-12-06T21:25:54.830Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 423.489837ms iteration = 23 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{"running":1},"pruned_instances":0,"total_instances":1,"update_sagas_queued":0}
8936	2024-12-06T21:25:54.840Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8937	2024-12-06T21:25:54.844Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
8938	2024-12-06T21:25:54.844Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
8939	2024-12-06T21:25:54.844Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
8940	2024-12-06T21:25:54.847Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 93c3f865-bc00-42df-bef7-20e39342f84f dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
8941	2024-12-06T21:25:54.847Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
8942	2024-12-06T21:25:54.847Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-06 21:15:06.025861 UTC dns_group = internal generation = 1
8943	2024-12-06T21:25:54.847Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 439.865814ms iteration = 12 status = {"generation":1}
8944	2024-12-06T21:25:54.851Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8945	2024-12-06T21:25:54.853Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
8946	2024-12-06T21:25:54.853Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8947	2024-12-06T21:25:54.854Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8948	2024-12-06T21:25:54.876Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8949	2024-12-06T21:25:54.876Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8950	2024-12-06T21:25:54.879Z	WARN	nexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
8951	2024-12-06T21:25:54.879Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 471.641716ms iteration = 12 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
8952	2024-12-06T21:25:54.914Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
8953	2024-12-06T21:25:54.914Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "d8978925-0e51-4273-bc33-deff870f1c6f", "date": "Fri, 06 Dec 2024 21:25:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
8954	2024-12-06T21:25:54.914Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
8955	2024-12-06T21:25:54.917Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "d92054bf-b2a0-4995-a147-d3cea717113b", "date": "Fri, 06 Dec 2024 21:25:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
8956	2024-12-06T21:25:54.917Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
8957	2024-12-06T21:25:54.918Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "93211593-c043-4052-9fd0-430f48a49e29", "date": "Fri, 06 Dec 2024 21:25:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
8958	2024-12-06T21:25:54.918Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
8959	2024-12-06T21:25:54.919Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
8960	2024-12-06T21:25:54.919Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 512.921683ms iteration = 14 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
8961	2024-12-06T21:25:54.998Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
8962	2024-12-06T21:25:54.998Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
8963	2024-12-06T21:25:54.998Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc051ea7-19e5-41e0-93ea-15175f219ad7", "content-length": "583", "date": "Fri, 06 Dec 2024 21:25:54 GMT"} })
8964	2024-12-06T21:25:54.998Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8965	2024-12-06T21:25:54.998Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8966	2024-12-06T21:25:55.000Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 05c53cbc-f66c-491f-af8d-6f291fe21596 dns_zone_name = oxide.test found_so_far = 1 version = 2
8967	2024-12-06T21:25:55.000Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
8968	2024-12-06T21:25:55.000Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-06 21:15:07.045426 UTC dns_group = external generation = 2
8969	2024-12-06T21:25:55.000Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 593.160179ms iteration = 12 status = {"generation":2}
8970	2024-12-06T21:25:55.028Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8971	2024-12-06T21:25:55.030Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
8972	2024-12-06T21:25:55.030Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8973	2024-12-06T21:25:55.030Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8974	2024-12-06T21:25:55.032Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8975	2024-12-06T21:25:55.034Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
8976	2024-12-06T21:25:55.034Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8977	2024-12-06T21:25:55.034Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8978	2024-12-06T21:25:55.038Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8979	2024-12-06T21:25:55.038Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8980	2024-12-06T21:25:55.040Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 3d56b7bf-53ed-4147-b45b-9cef154d7338, rack_id: 2d75e4a3-6363-4fd1-935e-989d31fd7d71, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(02859c9b-92d7-4b19-b425-8f266763be54) } rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
8981	2024-12-06T21:25:55.040Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8982	2024-12-06T21:25:55.040Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8983	2024-12-06T21:25:55.049Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {ce30fdf2-f481-4444-a53b-35bded50bb2c: {10.151.2.175: {8a0a43ee-1fc8-4092-9c8c-ea2df63f15f7}, 10.151.2.176: {8a0a43ee-1fc8-4092-9c8c-ea2df63f15f7}}} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
8984	2024-12-06T21:25:55.049Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
8985	2024-12-06T21:25:55.049Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
8986	2024-12-06T21:25:55.050Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "4a1a0324-863d-4590-a907-42a7bacd1215", "date": "Fri, 06 Dec 2024 21:25:54 GMT"} })
8987	2024-12-06T21:25:55.050Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8988	2024-12-06T21:25:55.050Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8989	2024-12-06T21:25:55.050Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e4a8650-62dd-4738-9786-ba98fce923f1", "content-length": "134", "date": "Fri, 06 Dec 2024 21:25:54 GMT"} })
8990	2024-12-06T21:25:55.050Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
8991	2024-12-06T21:25:55.050Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 02859c9b-92d7-4b19-b425-8f266763be54
8992	2024-12-06T21:25:55.050Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
8993	2024-12-06T21:25:55.050Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
8994	2024-12-06T21:25:55.050Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8995	2024-12-06T21:25:55.050Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8996	2024-12-06T21:25:55.050Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8997	2024-12-06T21:25:55.050Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
8998	2024-12-06T21:25:55.051Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8999	2024-12-06T21:25:55.051Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
9000	2024-12-06T21:25:55.051Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
9001	2024-12-06T21:25:55.051Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9002	2024-12-06T21:25:55.051Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9003	2024-12-06T21:25:55.052Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9004	2024-12-06T21:25:55.052Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9005	2024-12-06T21:25:55.053Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
9006	2024-12-06T21:25:55.053Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "f0f46f13-55f9-434b-9290-d4e75287fb98", "content-length": "2", "date": "Fri, 06 Dec 2024 21:25:54 GMT"} })
9007	2024-12-06T21:25:55.053Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
9008	2024-12-06T21:25:55.053Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "ecd51e36-c431-4bb0-9777-2b3d96e05cac", "content-length": "52", "date": "Fri, 06 Dec 2024 21:25:54 GMT"} })
9009	2024-12-06T21:25:55.053Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
9010	2024-12-06T21:25:55.053Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
9011	2024-12-06T21:25:55.055Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9012	2024-12-06T21:25:55.055Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9013	2024-12-06T21:25:55.058Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9014	2024-12-06T21:25:55.058Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9015	2024-12-06T21:25:55.060Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9016	2024-12-06T21:25:55.061Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9017	2024-12-06T21:25:55.063Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9018	2024-12-06T21:25:55.064Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9019	2024-12-06T21:25:55.071Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
9020	2024-12-06T21:25:55.071Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
9021	2024-12-06T21:25:55.071Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
9022	2024-12-06T21:25:55.128Z	DEBG	nexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
9023	2024-12-06T21:25:55.128Z	DEBG	nexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
9024	2024-12-06T21:25:55.136Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
9025	2024-12-06T21:25:55.158Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "ccd7355a-c0a7-413c-b24c-a0ac86aac53e", "date": "Fri, 06 Dec 2024 21:25:55 GMT"} })
9026	2024-12-06T21:25:55.158Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch = Switch0
9027	2024-12-06T21:25:55.158Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
9028	2024-12-06T21:25:55.159Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "24fb12cf-73d5-4bd2-8c7a-d9af4fe81041", "content-length": "682", "date": "Fri, 06 Dec 2024 21:25:55 GMT"} })
9029	2024-12-06T21:25:55.159Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9030	2024-12-06T21:25:55.159Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9031	2024-12-06T21:25:55.159Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "e6df2085-b078-498d-8556-9a270db88ed2", "date": "Fri, 06 Dec 2024 21:25:55 GMT"} })
9032	2024-12-06T21:25:55.159Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b7d2156-2518-4592-9def-898e3cc3d7b5", "content-length": "810", "date": "Fri, 06 Dec 2024 21:25:55 GMT"} })
9033	2024-12-06T21:25:55.159Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {VirtualNetworkInterfaceHost { physical_host_ip: fd00:1122:3344:101::1, virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), vni: Vni(1298706) }} sled = sock
9034	2024-12-06T21:25:55.159Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
9035	2024-12-06T21:25:55.159Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
9036	2024-12-06T21:25:55.162Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 754.877601ms iteration = 23 status = {}
9037	2024-12-06T21:25:55.162Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 754.960873ms iteration = 23 status = {}
9038	2024-12-06T21:25:55.164Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9039	2024-12-06T21:25:55.164Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9040	2024-12-06T21:25:55.165Z	DEBG	nexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
9041	2024-12-06T21:25:55.165Z	INFO	nexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:38520
9042	2024-12-06T21:25:55.165Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:38520 req_id = 18100907-5c16-496c-abd3-4b1c033e85c4 roles = RoleSet { roles: {} } uri = /metrics/producers
9043	2024-12-06T21:25:55.165Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:38520 req_id = 18100907-5c16-496c-abd3-4b1c033e85c4 resource = Database result = Ok(()) uri = /metrics/producers
9044	2024-12-06T21:25:55.169Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9045	2024-12-06T21:25:55.169Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9046	2024-12-06T21:25:55.171Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9047	2024-12-06T21:25:55.172Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9048	2024-12-06T21:25:55.181Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 774.406082ms iteration = 23 status = {}
9049	2024-12-06T21:25:55.212Z	INFO	nexus: assigned collector to new producer collector_id = 9d509eff-9671-46fb-8274-6ec9019a9d9e file = nexus/src/app/oximeter.rs:76 producer_id = e8725c8c-13ab-4916-bf16-bb7f832014d5
9050	2024-12-06T21:25:55.212Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 47484 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:38520 req_id = 18100907-5c16-496c-abd3-4b1c033e85c4 response_code = 201 uri = /metrics/producers
9051	2024-12-06T21:25:55.212Z	DEBG	nexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "18100907-5c16-496c-abd3-4b1c033e85c4", "content-length": "41", "date": "Fri, 06 Dec 2024 21:25:55 GMT"} })
9052	2024-12-06T21:25:55.214Z	DEBG	nexus (producer-registration-task): registered with nexus successfully
9053	2024-12-06T21:25:55.214Z	DEBG	nexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
9054	2024-12-06T21:25:56.541Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 314 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 320de7d6-6f7b-4907-8fe1-271d8c06b92e response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
9055	2024-12-06T21:26:04.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 68 reason = Timeout
9056	2024-12-06T21:26:04.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 68 reason = Timeout
9057	2024-12-06T21:26:04.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9058	2024-12-06T21:26:04.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9059	2024-12-06T21:26:04.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9060	2024-12-06T21:26:04.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9061	2024-12-06T21:26:04.437Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9062	2024-12-06T21:26:04.440Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9063	2024-12-06T21:26:04.440Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9064	2024-12-06T21:26:04.440Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9065	2024-12-06T21:26:04.443Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9066	2024-12-06T21:26:04.443Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9067	2024-12-06T21:26:04.443Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9068	2024-12-06T21:26:04.443Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9069	2024-12-06T21:26:04.444Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9070	2024-12-06T21:26:04.447Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
9071	2024-12-06T21:26:04.447Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9072	2024-12-06T21:26:04.447Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9073	2024-12-06T21:26:04.459Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
9074	2024-12-06T21:26:04.460Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 52.170546ms iteration = 68 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
9075	2024-12-06T21:26:04.473Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 65.421443ms iteration = 68 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
9076	2024-12-06T21:26:06.542Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 323 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 216c5d77-b74e-4431-92b2-6df336be649c response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
9077	2024-12-06T21:26:14.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 69 reason = Timeout
9078	2024-12-06T21:26:14.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 69 reason = Timeout
9079	2024-12-06T21:26:14.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9080	2024-12-06T21:26:14.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9081	2024-12-06T21:26:14.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9082	2024-12-06T21:26:14.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9083	2024-12-06T21:26:14.409Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9084	2024-12-06T21:26:14.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9085	2024-12-06T21:26:14.412Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9086	2024-12-06T21:26:14.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9087	2024-12-06T21:26:14.413Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9088	2024-12-06T21:26:14.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9089	2024-12-06T21:26:14.415Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9090	2024-12-06T21:26:14.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
9091	2024-12-06T21:26:14.418Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9092	2024-12-06T21:26:14.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9093	2024-12-06T21:26:14.428Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
9094	2024-12-06T21:26:14.428Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.046255ms iteration = 69 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
9095	2024-12-06T21:26:14.437Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9096	2024-12-06T21:26:14.437Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9097	2024-12-06T21:26:14.439Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.079516ms iteration = 69 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
9098	2024-12-06T21:26:16.542Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 365 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = d3246847-1592-4e2c-906c-3153a0d32739 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
9099	2024-12-06T21:26:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 70 reason = Timeout
9100	2024-12-06T21:26:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 24 reason = Timeout
9101	2024-12-06T21:26:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
9102	2024-12-06T21:26:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9103	2024-12-06T21:26:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 26 reason = Timeout
9104	2024-12-06T21:26:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 24 reason = Timeout
9105	2024-12-06T21:26:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
9106	2024-12-06T21:26:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
9107	2024-12-06T21:26:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 24 reason = Timeout
9108	2024-12-06T21:26:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9109	2024-12-06T21:26:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 24 reason = Timeout
9110	2024-12-06T21:26:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 24 reason = Timeout
9111	2024-12-06T21:26:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
9112	2024-12-06T21:26:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 24 reason = Timeout
9113	2024-12-06T21:26:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
9114	2024-12-06T21:26:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 24 reason = Timeout
9115	2024-12-06T21:26:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 24 reason = Timeout
9116	2024-12-06T21:26:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
9117	2024-12-06T21:26:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
9118	2024-12-06T21:26:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 24 reason = Timeout
9119	2024-12-06T21:26:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
9120	2024-12-06T21:26:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 24 reason = Timeout
9121	2024-12-06T21:26:24.407Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
9122	2024-12-06T21:26:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 24 reason = Timeout
9123	2024-12-06T21:26:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9124	2024-12-06T21:26:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
9125	2024-12-06T21:26:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 24 reason = Timeout
9126	2024-12-06T21:26:24.407Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
9127	2024-12-06T21:26:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
9128	2024-12-06T21:26:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 70 reason = Timeout
9129	2024-12-06T21:26:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9130	2024-12-06T21:26:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 24 reason = Timeout
9131	2024-12-06T21:26:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 24 reason = Timeout
9132	2024-12-06T21:26:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9133	2024-12-06T21:26:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
9134	2024-12-06T21:26:24.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9135	2024-12-06T21:26:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
9136	2024-12-06T21:26:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9137	2024-12-06T21:26:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
9138	2024-12-06T21:26:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
9139	2024-12-06T21:26:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
9140	2024-12-06T21:26:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
9141	2024-12-06T21:26:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
9142	2024-12-06T21:26:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
9143	2024-12-06T21:26:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9144	2024-12-06T21:26:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
9145	2024-12-06T21:26:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
9146	2024-12-06T21:26:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
9147	2024-12-06T21:26:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9148	2024-12-06T21:26:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
9149	2024-12-06T21:26:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9150	2024-12-06T21:26:24.410Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9151	2024-12-06T21:26:24.411Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9152	2024-12-06T21:26:24.412Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.444433ms iteration = 24 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
9153	2024-12-06T21:26:24.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
9154	2024-12-06T21:26:24.413Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9155	2024-12-06T21:26:24.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9156	2024-12-06T21:26:24.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9157	2024-12-06T21:26:24.413Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9158	2024-12-06T21:26:24.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9159	2024-12-06T21:26:24.421Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1499, tv_nsec: 919229305 } })
9160	2024-12-06T21:26:24.422Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
9161	2024-12-06T21:26:24.423Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9162	2024-12-06T21:26:24.426Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
9163	2024-12-06T21:26:24.426Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9164	2024-12-06T21:26:24.426Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1499, tv_nsec: 923987423 } })
9165	2024-12-06T21:26:24.426Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9166	2024-12-06T21:26:24.426Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
9167	2024-12-06T21:26:24.429Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9168	2024-12-06T21:26:24.430Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9169	2024-12-06T21:26:24.431Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9170	2024-12-06T21:26:24.435Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1499, tv_nsec: 933207824 } })
9171	2024-12-06T21:26:24.435Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
9172	2024-12-06T21:26:24.435Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9173	2024-12-06T21:26:24.436Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
9174	2024-12-06T21:26:24.436Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9175	2024-12-06T21:26:24.453Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
9176	2024-12-06T21:26:24.453Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
9177	2024-12-06T21:26:24.458Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9178	2024-12-06T21:26:24.458Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
9179	2024-12-06T21:26:24.458Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 50.971756ms iteration = 70 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
9180	2024-12-06T21:26:24.459Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
9181	2024-12-06T21:26:24.460Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
9182	2024-12-06T21:26:24.461Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
9183	2024-12-06T21:26:24.461Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
9184	2024-12-06T21:26:24.461Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
9185	2024-12-06T21:26:24.464Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
9186	2024-12-06T21:26:24.464Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 56.547168ms iteration = 24 status = {"errors":[],"records_set_to_done":[]}
9187	2024-12-06T21:26:24.464Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
9188	2024-12-06T21:26:24.464Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 57.207518ms iteration = 24 status = {"errors":[],"garbage_collect_requested":[]}
9189	2024-12-06T21:26:24.465Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9190	2024-12-06T21:26:24.465Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9191	2024-12-06T21:26:24.465Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9192	2024-12-06T21:26:24.466Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
9193	2024-12-06T21:26:24.466Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
9194	2024-12-06T21:26:24.466Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
9195	2024-12-06T21:26:24.466Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
9196	2024-12-06T21:26:24.466Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
9197	2024-12-06T21:26:24.466Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
9198	2024-12-06T21:26:24.466Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9199	2024-12-06T21:26:24.467Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9200	2024-12-06T21:26:24.467Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
9201	2024-12-06T21:26:24.467Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
9202	2024-12-06T21:26:24.467Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
9203	2024-12-06T21:26:24.468Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 60.611624ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
9204	2024-12-06T21:26:24.468Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
9205	2024-12-06T21:26:24.468Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
9206	2024-12-06T21:26:24.468Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
9207	2024-12-06T21:26:24.471Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
9208	2024-12-06T21:26:24.471Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9209	2024-12-06T21:26:24.471Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9210	2024-12-06T21:26:24.473Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9211	2024-12-06T21:26:24.473Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9212	2024-12-06T21:26:24.473Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9213	2024-12-06T21:26:24.475Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
9214	2024-12-06T21:26:24.475Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9215	2024-12-06T21:26:24.475Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9216	2024-12-06T21:26:24.476Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 1 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
9217	2024-12-06T21:26:24.476Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 68.534395ms iteration = 24 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":1,"total_instances":0,"update_sagas_queued":0}
9218	2024-12-06T21:26:24.487Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 79.605595ms iteration = 24 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
9219	2024-12-06T21:26:24.494Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
9220	2024-12-06T21:26:24.494Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 86.59979ms iteration = 70 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
9221	2024-12-06T21:26:24.494Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
9222	2024-12-06T21:26:24.497Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9223	2024-12-06T21:26:24.497Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9224	2024-12-06T21:26:24.499Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9225	2024-12-06T21:26:24.499Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9226	2024-12-06T21:26:24.500Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
9227	2024-12-06T21:26:24.500Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
9228	2024-12-06T21:26:24.500Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
9229	2024-12-06T21:26:24.501Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
9230	2024-12-06T21:26:24.501Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9231	2024-12-06T21:26:24.502Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
9232	2024-12-06T21:26:24.502Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
9233	2024-12-06T21:26:24.502Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9600934-3810-45cd-ab1f-236129c1012e", "content-length": "26", "date": "Fri, 06 Dec 2024 21:26:24 GMT"} })
9234	2024-12-06T21:26:24.502Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
9235	2024-12-06T21:26:24.503Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cbc5c1c6-5668-455a-85ef-94f4aebe0f80, lookup_type: ById(cbc5c1c6-5668-455a-85ef-94f4aebe0f80) } result = Ok(())
9236	2024-12-06T21:26:24.503Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9237	2024-12-06T21:26:24.504Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9238	2024-12-06T21:26:24.504Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
9239	2024-12-06T21:26:24.506Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 98.273882ms iteration = 24 status = 0
9240	2024-12-06T21:26:24.507Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
9241	2024-12-06T21:26:24.507Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
9242	2024-12-06T21:26:24.512Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
9243	2024-12-06T21:26:24.512Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
9244	2024-12-06T21:26:24.512Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "92b81a7c-c349-4431-948a-429215d06943", "content-length": "26", "date": "Fri, 06 Dec 2024 21:26:24 GMT"} })
9245	2024-12-06T21:26:24.512Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
9246	2024-12-06T21:26:24.514Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
9247	2024-12-06T21:26:24.515Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
9248	2024-12-06T21:26:24.516Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
9249	2024-12-06T21:26:24.516Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 zone_address = fd00:1122:3344:101::2
9250	2024-12-06T21:26:24.517Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
9251	2024-12-06T21:26:24.517Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "bfd098eb-aac6-4ccd-a3f9-d75597b5d9f6", "content-length": "26", "date": "Fri, 06 Dec 2024 21:26:24 GMT"} })
9252	2024-12-06T21:26:24.517Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
9253	2024-12-06T21:26:24.517Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9254	2024-12-06T21:26:24.518Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
9255	2024-12-06T21:26:24.519Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
9256	2024-12-06T21:26:24.519Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
9257	2024-12-06T21:26:24.519Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
9258	2024-12-06T21:26:24.521Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9259	2024-12-06T21:26:24.523Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
9260	2024-12-06T21:26:24.523Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
9261	2024-12-06T21:26:24.523Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
9262	2024-12-06T21:26:24.535Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
9263	2024-12-06T21:26:24.535Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
9264	2024-12-06T21:26:24.535Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 127.937628ms iteration = 24 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
9265	2024-12-06T21:26:24.536Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 129.300021ms iteration = 24 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
9266	2024-12-06T21:26:24.580Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 173.355334ms iteration = 26 status = {"physical_disks_added":0}
9267	2024-12-06T21:26:24.625Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
9268	2024-12-06T21:26:24.625Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a97a57f-80d7-4d5b-83de-eac341ad49d4", "content-length": "2", "date": "Fri, 06 Dec 2024 21:26:23 GMT"} })
9269	2024-12-06T21:26:24.627Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
9270	2024-12-06T21:26:24.627Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
9271	2024-12-06T21:26:24.630Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 222.395128ms iteration = 24 status = null
9272	2024-12-06T21:26:24.653Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
9273	2024-12-06T21:26:24.653Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "713c5b9f-c210-452c-9b7a-2759720758cb", "content-length": "691", "date": "Fri, 06 Dec 2024 21:26:24 GMT"} })
9274	2024-12-06T21:26:24.653Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
9275	2024-12-06T21:26:24.653Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
9276	2024-12-06T21:26:24.653Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
9277	2024-12-06T21:26:24.655Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 247.611141ms iteration = 24 status = {}
9278	2024-12-06T21:26:24.661Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
9279	2024-12-06T21:26:24.661Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
9280	2024-12-06T21:26:24.661Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "92eda8eb-153c-47bf-b77f-688576e971d8", "content-length": "372", "date": "Fri, 06 Dec 2024 21:26:24 GMT"} })
9281	2024-12-06T21:26:24.661Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9282	2024-12-06T21:26:24.661Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9283	2024-12-06T21:26:24.676Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
9284	2024-12-06T21:26:24.676Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
9285	2024-12-06T21:26:24.676Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "b46f2aea-1327-43e7-8981-e9f99dae0fea", "date": "Fri, 06 Dec 2024 21:26:24 GMT"} })
9286	2024-12-06T21:26:24.676Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9287	2024-12-06T21:26:24.677Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9288	2024-12-06T21:26:24.678Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9289	2024-12-06T21:26:24.679Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9290	2024-12-06T21:26:24.682Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9291	2024-12-06T21:26:24.682Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9292	2024-12-06T21:26:24.685Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
9293	2024-12-06T21:26:24.685Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
9294	2024-12-06T21:26:24.685Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "6c6ccaa5-d79a-41d6-9002-8bcebbdc01cf", "date": "Fri, 06 Dec 2024 21:26:24 GMT"} })
9295	2024-12-06T21:26:24.690Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 283.130654ms iteration = 24 status = {}
9296	2024-12-06T21:26:24.698Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 291.197376ms iteration = 24 status = {}
9297	2024-12-06T21:26:24.760Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9298	2024-12-06T21:26:24.760Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9299	2024-12-06T21:26:24.763Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 3d56b7bf-53ed-4147-b45b-9cef154d7338, rack_id: 2d75e4a3-6363-4fd1-935e-989d31fd7d71, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(02859c9b-92d7-4b19-b425-8f266763be54) } rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
9300	2024-12-06T21:26:24.763Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9301	2024-12-06T21:26:24.763Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9302	2024-12-06T21:26:24.773Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
9303	2024-12-06T21:26:24.774Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a7c7a06-8dc1-43b8-9224-7f5e64177db9", "content-length": "134", "date": "Fri, 06 Dec 2024 21:26:24 GMT"} })
9304	2024-12-06T21:26:24.774Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
9305	2024-12-06T21:26:24.774Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 02859c9b-92d7-4b19-b425-8f266763be54
9306	2024-12-06T21:26:24.774Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
9307	2024-12-06T21:26:24.774Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
9308	2024-12-06T21:26:24.774Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
9309	2024-12-06T21:26:24.774Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
9310	2024-12-06T21:26:24.774Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
9311	2024-12-06T21:26:24.774Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
9312	2024-12-06T21:26:24.774Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
9313	2024-12-06T21:26:24.774Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
9314	2024-12-06T21:26:24.774Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
9315	2024-12-06T21:26:24.774Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9316	2024-12-06T21:26:24.775Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9317	2024-12-06T21:26:24.777Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
9318	2024-12-06T21:26:24.777Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7d3823a-edc3-423e-a734-b09ee59a3e19", "content-length": "2", "date": "Fri, 06 Dec 2024 21:26:24 GMT"} })
9319	2024-12-06T21:26:24.777Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
9320	2024-12-06T21:26:24.777Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "6b619cae-5d17-4549-bcfc-46d6a1627bda", "content-length": "52", "date": "Fri, 06 Dec 2024 21:26:24 GMT"} })
9321	2024-12-06T21:26:24.777Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
9322	2024-12-06T21:26:24.777Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
9323	2024-12-06T21:26:24.867Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "2a9edc32-967b-4394-b11d-3bb836305811", "date": "Fri, 06 Dec 2024 21:26:24 GMT"} })
9324	2024-12-06T21:26:24.867Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch = Switch0
9325	2024-12-06T21:26:24.867Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
9326	2024-12-06T21:26:24.867Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "208847a4-f6cc-4d41-b376-9fe17462f711", "content-length": "682", "date": "Fri, 06 Dec 2024 21:26:24 GMT"} })
9327	2024-12-06T21:26:24.867Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9328	2024-12-06T21:26:24.868Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9329	2024-12-06T21:26:24.871Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9330	2024-12-06T21:26:24.871Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9331	2024-12-06T21:26:24.874Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9332	2024-12-06T21:26:24.875Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9333	2024-12-06T21:26:24.877Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9334	2024-12-06T21:26:24.877Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9335	2024-12-06T21:26:24.883Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 476.320722ms iteration = 24 status = {}
9336	2024-12-06T21:26:26.542Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 281 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = f22fb53c-dfad-409a-8d09-179a4d29a8e6 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
9337	2024-12-06T21:26:34.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 71 reason = Timeout
9338	2024-12-06T21:26:34.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 71 reason = Timeout
9339	2024-12-06T21:26:34.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9340	2024-12-06T21:26:34.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9341	2024-12-06T21:26:34.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9342	2024-12-06T21:26:34.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9343	2024-12-06T21:26:34.409Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9344	2024-12-06T21:26:34.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9345	2024-12-06T21:26:34.412Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9346	2024-12-06T21:26:34.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9347	2024-12-06T21:26:34.414Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9348	2024-12-06T21:26:34.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9349	2024-12-06T21:26:34.416Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9350	2024-12-06T21:26:34.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
9351	2024-12-06T21:26:34.418Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9352	2024-12-06T21:26:34.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9353	2024-12-06T21:26:34.431Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
9354	2024-12-06T21:26:34.431Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.144046ms iteration = 71 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
9355	2024-12-06T21:26:34.437Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9356	2024-12-06T21:26:34.438Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9357	2024-12-06T21:26:34.439Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.508764ms iteration = 71 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
9358	2024-12-06T21:26:36.542Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 289 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = b40cdd5d-3026-4474-b97c-4ec0bd082866 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
9359	2024-12-06T21:26:44.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 72 reason = Timeout
9360	2024-12-06T21:26:44.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 72 reason = Timeout
9361	2024-12-06T21:26:44.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9362	2024-12-06T21:26:44.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9363	2024-12-06T21:26:44.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9364	2024-12-06T21:26:44.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9365	2024-12-06T21:26:44.409Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9366	2024-12-06T21:26:44.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9367	2024-12-06T21:26:44.410Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9368	2024-12-06T21:26:44.411Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.923064ms iteration = 72 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
9369	2024-12-06T21:26:44.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9370	2024-12-06T21:26:44.412Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9371	2024-12-06T21:26:44.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9372	2024-12-06T21:26:44.414Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9373	2024-12-06T21:26:44.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9374	2024-12-06T21:26:44.416Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9375	2024-12-06T21:26:44.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
9376	2024-12-06T21:26:44.418Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9377	2024-12-06T21:26:44.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9378	2024-12-06T21:26:44.428Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
9379	2024-12-06T21:26:44.428Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.858562ms iteration = 72 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
9380	2024-12-06T21:26:46.542Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 292 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = f808459e-e960-41b5-84ca-179ffb65a553 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
9381	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 15 reason = Timeout
9382	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 15 reason = Timeout
9383	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 25 reason = Timeout
9384	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 73 reason = Timeout
9385	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 25 reason = Timeout
9386	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9387	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9388	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 25 reason = Timeout
9389	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
9390	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 14 reason = Timeout
9391	2024-12-06T21:26:54.408Z	WARN	nexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0
9392	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = external_endpoints iteration = 13 reason = Timeout
9393	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 9.01\u{b5}s iteration = 14 status = {"enabled":false,"target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0"}
9394	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 25 reason = Timeout
9395	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 13 reason = Timeout
9396	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
9397	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9398	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 25 reason = Timeout
9399	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9400	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 13 reason = Timeout
9401	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
9402	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 25 reason = Timeout
9403	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 6 reason = Timeout
9404	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
9405	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 13 reason = Timeout
9406	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9407	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
9408	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
9409	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 73 reason = Timeout
9410	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9411	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 25 reason = Timeout
9412	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 13 reason = Timeout
9413	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
9414	2024-12-06T21:26:54.408Z	INFO	nexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
9415	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 25 reason = Timeout
9416	2024-12-06T21:26:54.408Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
9417	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
9418	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9419	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 25 reason = Timeout
9420	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
9421	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 13 reason = Timeout
9422	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
9423	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 27 reason = Timeout
9424	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 25 reason = Timeout
9425	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
9426	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
9427	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 13 reason = Timeout
9428	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
9429	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
9430	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 13 reason = Timeout
9431	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 25 reason = Timeout
9432	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
9433	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 25 reason = Timeout
9434	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
9435	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 25 reason = Timeout
9436	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
9437	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
9438	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 13 reason = Timeout
9439	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 25 reason = Timeout
9440	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
9441	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
9442	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 13 reason = Timeout
9443	2024-12-06T21:26:54.408Z	INFO	nexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-06 21:16:54.408488148 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
9444	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
9445	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9446	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9447	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
9448	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9449	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
9450	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
9451	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9452	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
9453	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9454	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9455	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
9456	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
9457	2024-12-06T21:26:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
9458	2024-12-06T21:26:54.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
9459	2024-12-06T21:26:54.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
9460	2024-12-06T21:26:54.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9461	2024-12-06T21:26:54.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9462	2024-12-06T21:26:54.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
9463	2024-12-06T21:26:54.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
9464	2024-12-06T21:26:54.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
9465	2024-12-06T21:26:54.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
9466	2024-12-06T21:26:54.409Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4aff1ae3-2d71-40bf-bc46-79a7276da3c2.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4dccdb26-9cf9-40aa-9229-5d35905cf0e1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("79d475dc-44ec-44f4-9e51-ea8a753c39f3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("79d475dc-44ec-44f4-9e51-ea8a753c39f3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1529, tv_nsec: 906404577 } })
9467	2024-12-06T21:26:54.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
9468	2024-12-06T21:26:54.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
9469	2024-12-06T21:26:54.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
9470	2024-12-06T21:26:54.409Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("709da000-e0da-4b29-8160-3112fd080927.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a03854a6-7a78-402b-93b4-65c6d36bf236.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a03854a6-7a78-402b-93b4-65c6d36bf236.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1529, tv_nsec: 906439187 } })
9471	2024-12-06T21:26:54.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
9472	2024-12-06T21:26:54.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9473	2024-12-06T21:26:54.409Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9474	2024-12-06T21:26:54.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9475	2024-12-06T21:26:54.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
9476	2024-12-06T21:26:54.409Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9477	2024-12-06T21:26:54.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9478	2024-12-06T21:26:54.409Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9479	2024-12-06T21:26:54.409Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
9480	2024-12-06T21:26:54.409Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.279181ms iteration = 13 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
9481	2024-12-06T21:26:54.409Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
9482	2024-12-06T21:26:54.409Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.528815ms iteration = 13 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
9483	2024-12-06T21:26:54.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9484	2024-12-06T21:26:54.410Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9485	2024-12-06T21:26:54.411Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9486	2024-12-06T21:26:54.412Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.970235ms iteration = 25 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
9487	2024-12-06T21:26:54.412Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9488	2024-12-06T21:26:54.413Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1529, tv_nsec: 910387492 } })
9489	2024-12-06T21:26:54.413Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
9490	2024-12-06T21:26:54.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
9491	2024-12-06T21:26:54.413Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9492	2024-12-06T21:26:54.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9493	2024-12-06T21:26:54.414Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9494	2024-12-06T21:26:54.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9495	2024-12-06T21:26:54.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9496	2024-12-06T21:26:54.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
9497	2024-12-06T21:26:54.415Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9498	2024-12-06T21:26:54.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9499	2024-12-06T21:26:54.416Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9500	2024-12-06T21:26:54.416Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1529, tv_nsec: 913796978 } })
9501	2024-12-06T21:26:54.416Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
9502	2024-12-06T21:26:54.416Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9503	2024-12-06T21:26:54.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
9504	2024-12-06T21:26:54.416Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
9505	2024-12-06T21:26:54.417Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9506	2024-12-06T21:26:54.418Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
9507	2024-12-06T21:26:54.418Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9508	2024-12-06T21:26:54.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9509	2024-12-06T21:26:54.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
9510	2024-12-06T21:26:54.418Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
9511	2024-12-06T21:26:54.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
9512	2024-12-06T21:26:54.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
9513	2024-12-06T21:26:54.419Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9514	2024-12-06T21:26:54.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9515	2024-12-06T21:26:54.421Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
9516	2024-12-06T21:26:54.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
9517	2024-12-06T21:26:54.424Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9518	2024-12-06T21:26:54.425Z	DEBG	nexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 17.483527ms iteration = 13 status = {"errors":[],"found_port_ok":[]}
9519	2024-12-06T21:26:54.426Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
9520	2024-12-06T21:26:54.426Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
9521	2024-12-06T21:26:54.426Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
9522	2024-12-06T21:26:54.428Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9523	2024-12-06T21:26:54.428Z	INFO	nexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-06 21:16:54.408488148 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
9524	2024-12-06T21:26:54.428Z	DEBG	nexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 20.430265ms iteration = 13 status = {"expiration":"2024-12-06T21:16:54.408488148Z","pruned":[]}
9525	2024-12-06T21:26:54.430Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
9526	2024-12-06T21:26:54.430Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9527	2024-12-06T21:26:54.430Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9528	2024-12-06T21:26:54.430Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1529, tv_nsec: 928104783 } })
9529	2024-12-06T21:26:54.431Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
9530	2024-12-06T21:26:54.432Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9531	2024-12-06T21:26:54.432Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9532	2024-12-06T21:26:54.433Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9533	2024-12-06T21:26:54.433Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9534	2024-12-06T21:26:54.455Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9535	2024-12-06T21:26:54.457Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
9536	2024-12-06T21:26:54.457Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
9537	2024-12-06T21:26:54.458Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
9538	2024-12-06T21:26:54.459Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
9539	2024-12-06T21:26:54.459Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
9540	2024-12-06T21:26:54.459Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9541	2024-12-06T21:26:54.460Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9542	2024-12-06T21:26:54.461Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9543	2024-12-06T21:26:54.461Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
9544	2024-12-06T21:26:54.461Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9545	2024-12-06T21:26:54.461Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9546	2024-12-06T21:26:54.463Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9547	2024-12-06T21:26:54.464Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9548	2024-12-06T21:26:54.465Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
9549	2024-12-06T21:26:54.465Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
9550	2024-12-06T21:26:54.466Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
9551	2024-12-06T21:26:54.466Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
9552	2024-12-06T21:26:54.466Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
9553	2024-12-06T21:26:54.466Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
9554	2024-12-06T21:26:54.468Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
9555	2024-12-06T21:26:54.468Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9556	2024-12-06T21:26:54.468Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
9557	2024-12-06T21:26:54.468Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9558	2024-12-06T21:26:54.470Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
9559	2024-12-06T21:26:54.471Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
9560	2024-12-06T21:26:54.471Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9561	2024-12-06T21:26:54.472Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
9562	2024-12-06T21:26:54.472Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 64.405046ms iteration = 25 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
9563	2024-12-06T21:26:54.472Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 64.529218ms iteration = 25 status = {"errors":[],"records_set_to_done":[]}
9564	2024-12-06T21:26:54.472Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9565	2024-12-06T21:26:54.472Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9566	2024-12-06T21:26:54.473Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9567	2024-12-06T21:26:54.473Z	DEBG	nexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
9568	2024-12-06T21:26:54.473Z	INFO	nexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
9569	2024-12-06T21:26:54.473Z	DEBG	nexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 65.129768ms iteration = 13 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
9570	2024-12-06T21:26:54.474Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
9571	2024-12-06T21:26:54.474Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9572	2024-12-06T21:26:54.474Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9573	2024-12-06T21:26:54.474Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9574	2024-12-06T21:26:54.475Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
9575	2024-12-06T21:26:54.475Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9576	2024-12-06T21:26:54.475Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9577	2024-12-06T21:26:54.476Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 67.512887ms iteration = 25 status = {"errors":[],"garbage_collect_requested":[]}
9578	2024-12-06T21:26:54.476Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 68.485253ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
9579	2024-12-06T21:26:54.476Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9580	2024-12-06T21:26:54.476Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
9581	2024-12-06T21:26:54.477Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
9582	2024-12-06T21:26:54.477Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
9583	2024-12-06T21:26:54.477Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
9584	2024-12-06T21:26:54.478Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
9585	2024-12-06T21:26:54.479Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
9586	2024-12-06T21:26:54.479Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9587	2024-12-06T21:26:54.479Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9588	2024-12-06T21:26:54.479Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9589	2024-12-06T21:26:54.481Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
9590	2024-12-06T21:26:54.481Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
9591	2024-12-06T21:26:54.481Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
9592	2024-12-06T21:26:54.482Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
9593	2024-12-06T21:26:54.482Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
9594	2024-12-06T21:26:54.483Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 75.231054ms iteration = 25 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
9595	2024-12-06T21:26:54.484Z	DEBG	nexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 76.611756ms iteration = 6 status = {"nsuccess":5}
9596	2024-12-06T21:26:54.488Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
9597	2024-12-06T21:26:54.488Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
9598	2024-12-06T21:26:54.493Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
9599	2024-12-06T21:26:54.493Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 85.777677ms iteration = 73 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
9600	2024-12-06T21:26:54.507Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9601	2024-12-06T21:26:54.507Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9602	2024-12-06T21:26:54.509Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
9603	2024-12-06T21:26:54.509Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
9604	2024-12-06T21:26:54.509Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
9605	2024-12-06T21:26:54.509Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
9606	2024-12-06T21:26:54.509Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
9607	2024-12-06T21:26:54.509Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9608	2024-12-06T21:26:54.509Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9609	2024-12-06T21:26:54.510Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9610	2024-12-06T21:26:54.511Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9611	2024-12-06T21:26:54.511Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9612	2024-12-06T21:26:54.512Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9613	2024-12-06T21:26:54.513Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
9614	2024-12-06T21:26:54.513Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9615	2024-12-06T21:26:54.513Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9616	2024-12-06T21:26:54.513Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
9617	2024-12-06T21:26:54.513Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
9618	2024-12-06T21:26:54.513Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
9619	2024-12-06T21:26:54.513Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
9620	2024-12-06T21:26:54.514Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
9621	2024-12-06T21:26:54.514Z	DEBG	nexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
9622	2024-12-06T21:26:54.514Z	INFO	nexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
9623	2024-12-06T21:26:54.514Z	DEBG	nexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 106.202192ms iteration = 13 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
9624	2024-12-06T21:26:54.515Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cbc5c1c6-5668-455a-85ef-94f4aebe0f80, lookup_type: ById(cbc5c1c6-5668-455a-85ef-94f4aebe0f80) } result = Ok(())
9625	2024-12-06T21:26:54.515Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9626	2024-12-06T21:26:54.515Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
9627	2024-12-06T21:26:54.515Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
9628	2024-12-06T21:26:54.515Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
9629	2024-12-06T21:26:54.515Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9630	2024-12-06T21:26:54.515Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9631	2024-12-06T21:26:54.515Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
9632	2024-12-06T21:26:54.515Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9633	2024-12-06T21:26:54.516Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
9634	2024-12-06T21:26:54.516Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
9635	2024-12-06T21:26:54.516Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
9636	2024-12-06T21:26:54.516Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
9637	2024-12-06T21:26:54.518Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9638	2024-12-06T21:26:54.518Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 110.598094ms iteration = 25 status = 0
9639	2024-12-06T21:26:54.521Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 112.880802ms iteration = 73 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
9640	2024-12-06T21:26:54.521Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
9641	2024-12-06T21:26:54.521Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
9642	2024-12-06T21:26:54.521Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
9643	2024-12-06T21:26:54.525Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 116.635373ms iteration = 25 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
9644	2024-12-06T21:26:54.527Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
9645	2024-12-06T21:26:54.527Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
9646	2024-12-06T21:26:54.535Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 127.237617ms iteration = 27 status = {"physical_disks_added":0}
9647	2024-12-06T21:26:54.542Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
9648	2024-12-06T21:26:54.542Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
9649	2024-12-06T21:26:54.542Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a3888bc-9f43-4b2e-be07-510ca643c8c5", "content-length": "26", "date": "Fri, 06 Dec 2024 21:26:54 GMT"} })
9650	2024-12-06T21:26:54.542Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
9651	2024-12-06T21:26:54.543Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
9652	2024-12-06T21:26:54.544Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
9653	2024-12-06T21:26:54.544Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
9654	2024-12-06T21:26:54.544Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1a825e3-a468-4d30-ae8c-1d6a66d79352", "content-length": "26", "date": "Fri, 06 Dec 2024 21:26:54 GMT"} })
9655	2024-12-06T21:26:54.544Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
9656	2024-12-06T21:26:54.546Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
9657	2024-12-06T21:26:54.546Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
9658	2024-12-06T21:26:54.549Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
9659	2024-12-06T21:26:54.555Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9660	2024-12-06T21:26:54.555Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
9661	2024-12-06T21:26:54.555Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 05c53cbc-f66c-491f-af8d-6f291fe21596 dns_zone_name = oxide.test found_so_far = 0 version = 2
9662	2024-12-06T21:26:54.555Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
9663	2024-12-06T21:26:54.555Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9664	2024-12-06T21:26:54.555Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
9665	2024-12-06T21:26:54.557Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9666	2024-12-06T21:26:54.557Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9667	2024-12-06T21:26:54.559Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
9668	2024-12-06T21:26:54.559Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9669	2024-12-06T21:26:54.559Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9670	2024-12-06T21:26:54.560Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
9671	2024-12-06T21:26:54.560Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 93c3f865-bc00-42df-bef7-20e39342f84f dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
9672	2024-12-06T21:26:54.560Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
9673	2024-12-06T21:26:54.560Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
9674	2024-12-06T21:26:54.560Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
9675	2024-12-06T21:26:54.560Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
9676	2024-12-06T21:26:54.561Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
9677	2024-12-06T21:26:54.562Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9678	2024-12-06T21:26:54.563Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 05c53cbc-f66c-491f-af8d-6f291fe21596 dns_zone_name = oxide.test found_so_far = 1 version = 2
9679	2024-12-06T21:26:54.563Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
9680	2024-12-06T21:26:54.563Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-06 21:15:07.045426 UTC dns_group = external generation = 2
9681	2024-12-06T21:26:54.563Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 154.87639ms iteration = 13 status = {"generation":2}
9682	2024-12-06T21:26:54.564Z	WARN	nexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
9683	2024-12-06T21:26:54.564Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 156.04862ms iteration = 13 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
9684	2024-12-06T21:26:54.564Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
9685	2024-12-06T21:26:54.564Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
9686	2024-12-06T21:26:54.564Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 156.126881ms iteration = 25 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
9687	2024-12-06T21:26:54.565Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
9688	2024-12-06T21:26:54.565Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
9689	2024-12-06T21:26:54.565Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
9690	2024-12-06T21:26:54.567Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 93c3f865-bc00-42df-bef7-20e39342f84f dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
9691	2024-12-06T21:26:54.567Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
9692	2024-12-06T21:26:54.567Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-06 21:15:06.025861 UTC dns_group = internal generation = 1
9693	2024-12-06T21:26:54.567Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 159.332434ms iteration = 13 status = {"generation":1}
9694	2024-12-06T21:26:54.569Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9695	2024-12-06T21:26:54.571Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
9696	2024-12-06T21:26:54.571Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
9697	2024-12-06T21:26:54.571Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
9698	2024-12-06T21:26:54.573Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9699	2024-12-06T21:26:54.574Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
9700	2024-12-06T21:26:54.574Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
9701	2024-12-06T21:26:54.574Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
9702	2024-12-06T21:26:54.583Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 zone_address = fd00:1122:3344:101::2
9703	2024-12-06T21:26:54.583Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
9704	2024-12-06T21:26:54.583Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c45f37f-f63f-4a11-9e6d-1079a6af6349", "content-length": "26", "date": "Fri, 06 Dec 2024 21:26:54 GMT"} })
9705	2024-12-06T21:26:54.583Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
9706	2024-12-06T21:26:54.584Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
9707	2024-12-06T21:26:54.602Z	DEBG	nexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-06T21:14:50.566199Z, time_modified: 2024-12-06T21:14:50.566199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, db_silo: Silo { identity: SiloIdentity { id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-06T21:15:06.991365Z, time_modified: 2024-12-06T21:15:06.991365Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, db_silo: Silo { identity: SiloIdentity { id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-06T21:15:06.991365Z, time_modified: 2024-12-06T21:15:06.991365Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1" }] }) }
9708	2024-12-06T21:26:54.602Z	DEBG	nexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 194.267086ms iteration = 13 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"683d02e1-6ec8-48bf-98d0-db6c94785f80","tls_certs":["d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1"]}},"default_endpoint":{"silo_id":"683d02e1-6ec8-48bf-98d0-db6c94785f80","tls_certs":["d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
9709	2024-12-06T21:26:54.719Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
9710	2024-12-06T21:26:54.728Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
9711	2024-12-06T21:26:54.728Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "e19e3e0b-9bdd-4b7b-b42a-c297b3c7061b", "content-length": "2", "date": "Fri, 06 Dec 2024 21:26:53 GMT"} })
9712	2024-12-06T21:26:54.729Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
9713	2024-12-06T21:26:54.730Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
9714	2024-12-06T21:26:54.732Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 323.900173ms iteration = 25 status = null
9715	2024-12-06T21:26:54.732Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 324.448502ms iteration = 25 status = {}
9716	2024-12-06T21:26:54.733Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
9717	2024-12-06T21:26:54.733Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "854292f0-09eb-4605-b912-4404d3c86e1d", "date": "Fri, 06 Dec 2024 21:26:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
9718	2024-12-06T21:26:54.733Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
9719	2024-12-06T21:26:54.735Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "3fce9802-fef6-4f15-94ad-e5e5540c6ff9", "date": "Fri, 06 Dec 2024 21:26:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
9720	2024-12-06T21:26:54.735Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
9721	2024-12-06T21:26:54.737Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
9722	2024-12-06T21:26:54.737Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 329.577966ms iteration = 15 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
9723	2024-12-06T21:26:54.750Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
9724	2024-12-06T21:26:54.750Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
9725	2024-12-06T21:26:54.750Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "3a7ab091-ec0b-4380-a7e9-e3ffca1ff7e7", "content-length": "372", "date": "Fri, 06 Dec 2024 21:26:54 GMT"} })
9726	2024-12-06T21:26:54.750Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9727	2024-12-06T21:26:54.751Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9728	2024-12-06T21:26:54.786Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
9729	2024-12-06T21:26:54.787Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7a64678-94d8-40a1-b402-f26a322b9501", "content-length": "691", "date": "Fri, 06 Dec 2024 21:26:54 GMT"} })
9730	2024-12-06T21:26:54.787Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
9731	2024-12-06T21:26:54.787Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
9732	2024-12-06T21:26:54.787Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
9733	2024-12-06T21:26:54.788Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 380.539391ms iteration = 25 status = {}
9734	2024-12-06T21:26:54.798Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
9735	2024-12-06T21:26:54.798Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
9736	2024-12-06T21:26:54.798Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "ba62d8e6-07ac-48d7-bd1f-d3bf4df2ab9e", "date": "Fri, 06 Dec 2024 21:26:54 GMT"} })
9737	2024-12-06T21:26:54.798Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9738	2024-12-06T21:26:54.799Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9739	2024-12-06T21:26:54.801Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9740	2024-12-06T21:26:54.801Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9741	2024-12-06T21:26:54.804Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9742	2024-12-06T21:26:54.804Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9743	2024-12-06T21:26:54.807Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
9744	2024-12-06T21:26:54.807Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
9745	2024-12-06T21:26:54.808Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "378041d2-e028-4f45-b48e-95247eed81ed", "date": "Fri, 06 Dec 2024 21:26:54 GMT"} })
9746	2024-12-06T21:26:54.808Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
9747	2024-12-06T21:26:54.809Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "2265ace1-8086-4fe2-bd5c-311a58cb10d5", "date": "Fri, 06 Dec 2024 21:26:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
9748	2024-12-06T21:26:54.809Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
9749	2024-12-06T21:26:54.810Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 401.713119ms iteration = 25 status = {}
9750	2024-12-06T21:26:54.811Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "fe881d8a-fd9a-4440-8fff-8276298392d7", "date": "Fri, 06 Dec 2024 21:26:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
9751	2024-12-06T21:26:54.811Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
9752	2024-12-06T21:26:54.813Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "6623da29-2a5f-43d1-b9fd-0ebd312029dc", "date": "Fri, 06 Dec 2024 21:26:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
9753	2024-12-06T21:26:54.813Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
9754	2024-12-06T21:26:54.815Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
9755	2024-12-06T21:26:54.815Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 407.028686ms iteration = 15 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
9756	2024-12-06T21:26:54.851Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9757	2024-12-06T21:26:54.851Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9758	2024-12-06T21:26:54.853Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 3d56b7bf-53ed-4147-b45b-9cef154d7338, rack_id: 2d75e4a3-6363-4fd1-935e-989d31fd7d71, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(02859c9b-92d7-4b19-b425-8f266763be54) } rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
9759	2024-12-06T21:26:54.853Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9760	2024-12-06T21:26:54.853Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9761	2024-12-06T21:26:54.863Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
9762	2024-12-06T21:26:54.864Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "eca4257f-f925-4369-bae4-16927c40b6c1", "content-length": "134", "date": "Fri, 06 Dec 2024 21:26:54 GMT"} })
9763	2024-12-06T21:26:54.864Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
9764	2024-12-06T21:26:54.864Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 02859c9b-92d7-4b19-b425-8f266763be54
9765	2024-12-06T21:26:54.864Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
9766	2024-12-06T21:26:54.864Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
9767	2024-12-06T21:26:54.864Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
9768	2024-12-06T21:26:54.864Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
9769	2024-12-06T21:26:54.864Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
9770	2024-12-06T21:26:54.864Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
9771	2024-12-06T21:26:54.864Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
9772	2024-12-06T21:26:54.864Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
9773	2024-12-06T21:26:54.865Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
9774	2024-12-06T21:26:54.865Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9775	2024-12-06T21:26:54.865Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9776	2024-12-06T21:26:54.867Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
9777	2024-12-06T21:26:54.867Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "9f95e7a5-d9b0-49c3-b883-01fb61ca1015", "content-length": "2", "date": "Fri, 06 Dec 2024 21:26:54 GMT"} })
9778	2024-12-06T21:26:54.867Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
9779	2024-12-06T21:26:54.868Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "6021265b-0b9a-4af5-9810-4062217a080e", "content-length": "52", "date": "Fri, 06 Dec 2024 21:26:54 GMT"} })
9780	2024-12-06T21:26:54.868Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
9781	2024-12-06T21:26:54.868Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
9782	2024-12-06T21:26:54.985Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "0a1358ee-3e4d-49f7-8bc2-4c83dc9212a8", "date": "Fri, 06 Dec 2024 21:26:54 GMT"} })
9783	2024-12-06T21:26:54.985Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch = Switch0
9784	2024-12-06T21:26:54.985Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
9785	2024-12-06T21:26:54.985Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc1bb61b-0c68-4d47-9fe4-c88ad2badf5f", "content-length": "682", "date": "Fri, 06 Dec 2024 21:26:54 GMT"} })
9786	2024-12-06T21:26:54.985Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9787	2024-12-06T21:26:54.985Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9788	2024-12-06T21:26:54.990Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9789	2024-12-06T21:26:54.990Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9790	2024-12-06T21:26:54.993Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9791	2024-12-06T21:26:54.994Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9792	2024-12-06T21:26:54.996Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9793	2024-12-06T21:26:54.996Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9794	2024-12-06T21:26:55.003Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 594.922957ms iteration = 25 status = {}
9795	2024-12-06T21:26:56.542Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 303 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 1d3d334e-93e2-4b5c-8a18-b05a43a976ba response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
9796	2024-12-06T21:27:04.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 74 reason = Timeout
9797	2024-12-06T21:27:04.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 74 reason = Timeout
9798	2024-12-06T21:27:04.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9799	2024-12-06T21:27:04.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9800	2024-12-06T21:27:04.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9801	2024-12-06T21:27:04.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9802	2024-12-06T21:27:04.436Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9803	2024-12-06T21:27:04.436Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9804	2024-12-06T21:27:04.436Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9805	2024-12-06T21:27:04.438Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 30.676303ms iteration = 74 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
9806	2024-12-06T21:27:04.439Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9807	2024-12-06T21:27:04.439Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9808	2024-12-06T21:27:04.439Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9809	2024-12-06T21:27:04.441Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9810	2024-12-06T21:27:04.441Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9811	2024-12-06T21:27:04.443Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9812	2024-12-06T21:27:04.445Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
9813	2024-12-06T21:27:04.445Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9814	2024-12-06T21:27:04.445Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9815	2024-12-06T21:27:04.456Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
9816	2024-12-06T21:27:04.456Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 49.579884ms iteration = 74 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
9817	2024-12-06T21:27:06.541Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 298 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 77cb1779-0584-4a47-954e-d507ada8b117 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
9818	2024-12-06T21:27:14.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 75 reason = Timeout
9819	2024-12-06T21:27:14.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 75 reason = Timeout
9820	2024-12-06T21:27:14.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9821	2024-12-06T21:27:14.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9822	2024-12-06T21:27:14.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9823	2024-12-06T21:27:14.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9824	2024-12-06T21:27:14.409Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9825	2024-12-06T21:27:14.409Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9826	2024-12-06T21:27:14.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9827	2024-12-06T21:27:14.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9828	2024-12-06T21:27:14.411Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9829	2024-12-06T21:27:14.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9830	2024-12-06T21:27:14.413Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9831	2024-12-06T21:27:14.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9832	2024-12-06T21:27:14.415Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9833	2024-12-06T21:27:14.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
9834	2024-12-06T21:27:14.417Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9835	2024-12-06T21:27:14.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9836	2024-12-06T21:27:14.429Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
9837	2024-12-06T21:27:14.429Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.243625ms iteration = 75 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
9838	2024-12-06T21:27:14.439Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.819249ms iteration = 75 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
9839	2024-12-06T21:27:16.542Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 419 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = f0e9eaf4-f691-408e-b7b0-10b297ee5fd7 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
9840	2024-12-06T21:27:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 26 reason = Timeout
9841	2024-12-06T21:27:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
9842	2024-12-06T21:27:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 26 reason = Timeout
9843	2024-12-06T21:27:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 26 reason = Timeout
9844	2024-12-06T21:27:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
9845	2024-12-06T21:27:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
9846	2024-12-06T21:27:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 26 reason = Timeout
9847	2024-12-06T21:27:24.407Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
9848	2024-12-06T21:27:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 28 reason = Timeout
9849	2024-12-06T21:27:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9850	2024-12-06T21:27:24.407Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
9851	2024-12-06T21:27:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 26 reason = Timeout
9852	2024-12-06T21:27:24.407Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 26 reason = Timeout
9853	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
9854	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
9855	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
9856	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 26 reason = Timeout
9857	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
9858	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 26 reason = Timeout
9859	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
9860	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 26 reason = Timeout
9861	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
9862	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 26 reason = Timeout
9863	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9864	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 26 reason = Timeout
9865	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
9866	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 76 reason = Timeout
9867	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 26 reason = Timeout
9868	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9869	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 26 reason = Timeout
9870	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 26 reason = Timeout
9871	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9872	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 76 reason = Timeout
9873	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
9874	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9875	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
9876	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
9877	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
9878	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
9879	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
9880	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
9881	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
9882	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9883	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
9884	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
9885	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
9886	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
9887	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9888	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9889	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9890	2024-12-06T21:27:24.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9891	2024-12-06T21:27:24.411Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.076321ms iteration = 26 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
9892	2024-12-06T21:27:24.411Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1559, tv_nsec: 908888894 } })
9893	2024-12-06T21:27:24.411Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
9894	2024-12-06T21:27:24.412Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
9895	2024-12-06T21:27:24.412Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
9896	2024-12-06T21:27:24.412Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
9897	2024-12-06T21:27:24.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
9898	2024-12-06T21:27:24.413Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9899	2024-12-06T21:27:24.413Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9900	2024-12-06T21:27:24.413Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9901	2024-12-06T21:27:24.415Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
9902	2024-12-06T21:27:24.415Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9903	2024-12-06T21:27:24.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
9904	2024-12-06T21:27:24.415Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
9905	2024-12-06T21:27:24.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9906	2024-12-06T21:27:24.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
9907	2024-12-06T21:27:24.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
9908	2024-12-06T21:27:24.416Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
9909	2024-12-06T21:27:24.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
9910	2024-12-06T21:27:24.418Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9911	2024-12-06T21:27:24.418Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1559, tv_nsec: 915864849 } })
9912	2024-12-06T21:27:24.418Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
9913	2024-12-06T21:27:24.419Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9914	2024-12-06T21:27:24.420Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
9915	2024-12-06T21:27:24.420Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
9916	2024-12-06T21:27:24.420Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
9917	2024-12-06T21:27:24.420Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
9918	2024-12-06T21:27:24.420Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9919	2024-12-06T21:27:24.420Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9920	2024-12-06T21:27:24.420Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
9921	2024-12-06T21:27:24.422Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9922	2024-12-06T21:27:24.422Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9923	2024-12-06T21:27:24.422Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9924	2024-12-06T21:27:24.423Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9925	2024-12-06T21:27:24.425Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
9926	2024-12-06T21:27:24.425Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9927	2024-12-06T21:27:24.425Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9928	2024-12-06T21:27:24.430Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
9929	2024-12-06T21:27:24.430Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 22.647221ms iteration = 26 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
9930	2024-12-06T21:27:24.433Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9931	2024-12-06T21:27:24.433Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9932	2024-12-06T21:27:24.435Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
9933	2024-12-06T21:27:24.435Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1559, tv_nsec: 932656274 } })
9934	2024-12-06T21:27:24.435Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
9935	2024-12-06T21:27:24.435Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
9936	2024-12-06T21:27:24.437Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9937	2024-12-06T21:27:24.438Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
9938	2024-12-06T21:27:24.438Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
9939	2024-12-06T21:27:24.439Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
9940	2024-12-06T21:27:24.451Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 43.89355ms iteration = 26 status = {"errors":[],"garbage_collect_requested":[]}
9941	2024-12-06T21:27:24.453Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
9942	2024-12-06T21:27:24.453Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
9943	2024-12-06T21:27:24.454Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
9944	2024-12-06T21:27:24.455Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
9945	2024-12-06T21:27:24.456Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9946	2024-12-06T21:27:24.456Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
9947	2024-12-06T21:27:24.457Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
9948	2024-12-06T21:27:24.458Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 50.163353ms iteration = 26 status = {"errors":[],"records_set_to_done":[]}
9949	2024-12-06T21:27:24.458Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 50.820154ms iteration = 26 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
9950	2024-12-06T21:27:24.459Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
9951	2024-12-06T21:27:24.459Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9952	2024-12-06T21:27:24.459Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9953	2024-12-06T21:27:24.461Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9954	2024-12-06T21:27:24.461Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9955	2024-12-06T21:27:24.470Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
9956	2024-12-06T21:27:24.470Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
9957	2024-12-06T21:27:24.472Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
9958	2024-12-06T21:27:24.472Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
9959	2024-12-06T21:27:24.472Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 64.739682ms iteration = 26 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
9960	2024-12-06T21:27:24.474Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9961	2024-12-06T21:27:24.474Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9962	2024-12-06T21:27:24.476Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 67.896934ms iteration = 76 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
9963	2024-12-06T21:27:24.476Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
9964	2024-12-06T21:27:24.476Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
9965	2024-12-06T21:27:24.477Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c05fcc20-193e-4ad3-b350-e2b65818aff2", "content-length": "26", "date": "Fri, 06 Dec 2024 21:27:24 GMT"} })
9966	2024-12-06T21:27:24.477Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
9967	2024-12-06T21:27:24.478Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
9968	2024-12-06T21:27:24.478Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9969	2024-12-06T21:27:24.478Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9970	2024-12-06T21:27:24.480Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9971	2024-12-06T21:27:24.480Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9972	2024-12-06T21:27:24.481Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9973	2024-12-06T21:27:24.482Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9974	2024-12-06T21:27:24.485Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
9975	2024-12-06T21:27:24.485Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9976	2024-12-06T21:27:24.485Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9977	2024-12-06T21:27:24.488Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cbc5c1c6-5668-455a-85ef-94f4aebe0f80, lookup_type: ById(cbc5c1c6-5668-455a-85ef-94f4aebe0f80) } result = Ok(())
9978	2024-12-06T21:27:24.488Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9979	2024-12-06T21:27:24.488Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9980	2024-12-06T21:27:24.490Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 82.509764ms iteration = 26 status = 0
9981	2024-12-06T21:27:24.492Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 84.616458ms iteration = 26 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
9982	2024-12-06T21:27:24.496Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
9983	2024-12-06T21:27:24.496Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 88.669054ms iteration = 76 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
9984	2024-12-06T21:27:24.498Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
9985	2024-12-06T21:27:24.498Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
9986	2024-12-06T21:27:24.499Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c98443e5-c6e6-40c2-b745-49757735f1bf", "content-length": "26", "date": "Fri, 06 Dec 2024 21:27:24 GMT"} })
9987	2024-12-06T21:27:24.499Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
9988	2024-12-06T21:27:24.501Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
9989	2024-12-06T21:27:24.503Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 95.216531ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
9990	2024-12-06T21:27:24.507Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 zone_address = fd00:1122:3344:101::2
9991	2024-12-06T21:27:24.507Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
9992	2024-12-06T21:27:24.507Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a2668d7-fe42-499e-a951-f09a3bdadfdc", "content-length": "26", "date": "Fri, 06 Dec 2024 21:27:24 GMT"} })
9993	2024-12-06T21:27:24.507Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
9994	2024-12-06T21:27:24.509Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
9995	2024-12-06T21:27:24.541Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
9996	2024-12-06T21:27:24.541Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
9997	2024-12-06T21:27:24.554Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 146.091976ms iteration = 28 status = {"physical_disks_added":0}
9998	2024-12-06T21:27:24.575Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
9999	2024-12-06T21:27:24.575Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
10000	2024-12-06T21:27:24.577Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10001	2024-12-06T21:27:24.579Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
10002	2024-12-06T21:27:24.579Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
10003	2024-12-06T21:27:24.579Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
10004	2024-12-06T21:27:24.592Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
10005	2024-12-06T21:27:24.592Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "f1f3792c-e354-4e6a-ba77-ecf53cbb75d7", "content-length": "2", "date": "Fri, 06 Dec 2024 21:27:23 GMT"} })
10006	2024-12-06T21:27:24.594Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
10007	2024-12-06T21:27:24.594Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
10008	2024-12-06T21:27:24.610Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
10009	2024-12-06T21:27:24.610Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
10010	2024-12-06T21:27:24.611Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c3b4f7b-d137-4066-937c-e21edbb638db", "content-length": "372", "date": "Fri, 06 Dec 2024 21:27:23 GMT"} })
10011	2024-12-06T21:27:24.611Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10012	2024-12-06T21:27:24.611Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10013	2024-12-06T21:27:24.612Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10014	2024-12-06T21:27:24.613Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 205.308018ms iteration = 26 status = null
10015	2024-12-06T21:27:24.614Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
10016	2024-12-06T21:27:24.614Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
10017	2024-12-06T21:27:24.614Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
10018	2024-12-06T21:27:24.622Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 214.730772ms iteration = 26 status = {}
10019	2024-12-06T21:27:24.641Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
10020	2024-12-06T21:27:24.641Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
10021	2024-12-06T21:27:24.641Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "0f9acb9c-4609-48a6-b2af-ada58d6780ae", "date": "Fri, 06 Dec 2024 21:27:23 GMT"} })
10022	2024-12-06T21:27:24.641Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10023	2024-12-06T21:27:24.642Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10024	2024-12-06T21:27:24.644Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10025	2024-12-06T21:27:24.644Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10026	2024-12-06T21:27:24.647Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10027	2024-12-06T21:27:24.647Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10028	2024-12-06T21:27:24.651Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
10029	2024-12-06T21:27:24.651Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
10030	2024-12-06T21:27:24.651Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "eb8ad463-4a21-48a9-a649-81618bf54f8b", "date": "Fri, 06 Dec 2024 21:27:23 GMT"} })
10031	2024-12-06T21:27:24.653Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 245.238442ms iteration = 26 status = {}
10032	2024-12-06T21:27:24.718Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10033	2024-12-06T21:27:24.718Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10034	2024-12-06T21:27:24.736Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
10035	2024-12-06T21:27:24.737Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1c91931-1de3-4d97-93c2-6a092f2ba6b1", "content-length": "691", "date": "Fri, 06 Dec 2024 21:27:23 GMT"} })
10036	2024-12-06T21:27:24.737Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
10037	2024-12-06T21:27:24.737Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
10038	2024-12-06T21:27:24.737Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
10039	2024-12-06T21:27:24.738Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 330.154615ms iteration = 26 status = {}
10040	2024-12-06T21:27:24.738Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 3d56b7bf-53ed-4147-b45b-9cef154d7338, rack_id: 2d75e4a3-6363-4fd1-935e-989d31fd7d71, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(02859c9b-92d7-4b19-b425-8f266763be54) } rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
10041	2024-12-06T21:27:24.738Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10042	2024-12-06T21:27:24.739Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10043	2024-12-06T21:27:24.746Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
10044	2024-12-06T21:27:24.746Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "cf85814c-53e8-47ab-93a0-5c9ab6b727e5", "content-length": "134", "date": "Fri, 06 Dec 2024 21:27:23 GMT"} })
10045	2024-12-06T21:27:24.746Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
10046	2024-12-06T21:27:24.746Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 02859c9b-92d7-4b19-b425-8f266763be54
10047	2024-12-06T21:27:24.747Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
10048	2024-12-06T21:27:24.747Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
10049	2024-12-06T21:27:24.747Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
10050	2024-12-06T21:27:24.747Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
10051	2024-12-06T21:27:24.747Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
10052	2024-12-06T21:27:24.747Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
10053	2024-12-06T21:27:24.747Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
10054	2024-12-06T21:27:24.747Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
10055	2024-12-06T21:27:24.747Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
10056	2024-12-06T21:27:24.747Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10057	2024-12-06T21:27:24.747Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10058	2024-12-06T21:27:24.749Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
10059	2024-12-06T21:27:24.749Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a5bc13d-a40f-40bd-860a-387f51c9d344", "content-length": "2", "date": "Fri, 06 Dec 2024 21:27:23 GMT"} })
10060	2024-12-06T21:27:24.749Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
10061	2024-12-06T21:27:24.749Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "e8cf8198-2d08-4df8-9bdb-f0cb0fb884d8", "content-length": "52", "date": "Fri, 06 Dec 2024 21:27:23 GMT"} })
10062	2024-12-06T21:27:24.749Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
10063	2024-12-06T21:27:24.749Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
10064	2024-12-06T21:27:24.854Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "9fe39605-dc84-4e8d-9103-10bf9158eadd", "date": "Fri, 06 Dec 2024 21:27:24 GMT"} })
10065	2024-12-06T21:27:24.854Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch = Switch0
10066	2024-12-06T21:27:24.854Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
10067	2024-12-06T21:27:24.854Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "f3058000-a69a-4f4a-aa92-e0f6473db4d0", "content-length": "682", "date": "Fri, 06 Dec 2024 21:27:24 GMT"} })
10068	2024-12-06T21:27:24.854Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10069	2024-12-06T21:27:24.855Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10070	2024-12-06T21:27:24.858Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10071	2024-12-06T21:27:24.858Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10072	2024-12-06T21:27:24.861Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10073	2024-12-06T21:27:24.862Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10074	2024-12-06T21:27:24.863Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10075	2024-12-06T21:27:24.864Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10076	2024-12-06T21:27:24.869Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 461.830825ms iteration = 26 status = {}
10077	2024-12-06T21:27:26.542Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 296 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = 608aa445-1b87-46ae-bbf5-933d2af353ae response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
10078	2024-12-06T21:27:34.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 77 reason = Timeout
10079	2024-12-06T21:27:34.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 77 reason = Timeout
10080	2024-12-06T21:27:34.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10081	2024-12-06T21:27:34.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
10082	2024-12-06T21:27:34.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10083	2024-12-06T21:27:34.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
10084	2024-12-06T21:27:34.409Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10085	2024-12-06T21:27:34.410Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
10086	2024-12-06T21:27:34.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
10087	2024-12-06T21:27:34.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
10088	2024-12-06T21:27:34.412Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10089	2024-12-06T21:27:34.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10090	2024-12-06T21:27:34.412Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.86542ms iteration = 77 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
10091	2024-12-06T21:27:34.414Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10092	2024-12-06T21:27:34.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10093	2024-12-06T21:27:34.416Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10094	2024-12-06T21:27:34.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
10095	2024-12-06T21:27:34.418Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10096	2024-12-06T21:27:34.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10097	2024-12-06T21:27:34.427Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
10098	2024-12-06T21:27:34.427Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.143661ms iteration = 77 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
10099	2024-12-06T21:27:36.536Z	INFO	nexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:41659
10100	2024-12-06T21:27:36.536Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41659 req_id = 608f59b2-bfa3-49ce-a653-cae6014d6427 roles = RoleSet { roles: {} } uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
10101	2024-12-06T21:27:36.537Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41659 req_id = 608f59b2-bfa3-49ce-a653-cae6014d6427 resource = Database result = Ok(()) uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
10102	2024-12-06T21:27:36.539Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3466 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41659 req_id = 608f59b2-bfa3-49ce-a653-cae6014d6427 response_code = 200 uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?sort_by=id_ascending
10103	2024-12-06T21:27:36.540Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41659 req_id = 99cd93fb-317c-4e1c-b17f-54e5aac7c46c roles = RoleSet { roles: {} } uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
10104	2024-12-06T21:27:36.540Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41659 req_id = 99cd93fb-317c-4e1c-b17f-54e5aac7c46c resource = Database result = Ok(()) uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
10105	2024-12-06T21:27:36.542Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2295 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41659 req_id = 99cd93fb-317c-4e1c-b17f-54e5aac7c46c response_code = 200 uri = /metrics/collectors/9d509eff-9671-46fb-8274-6ec9019a9d9e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlODcyNWM4Yy0xM2FiLTQ5MTYtYmYxNi1iYjdmODMyMDE0ZDUifX0%3D
10106	2024-12-06T21:27:36.542Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 280 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = d2416e0e-b2ba-4d33-abbd-5664da1a6832 response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
10107	2024-12-06T21:27:44.407Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 78 reason = Timeout
10108	2024-12-06T21:27:44.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 78 reason = Timeout
10109	2024-12-06T21:27:44.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10110	2024-12-06T21:27:44.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
10111	2024-12-06T21:27:44.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10112	2024-12-06T21:27:44.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
10113	2024-12-06T21:27:44.409Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10114	2024-12-06T21:27:44.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
10115	2024-12-06T21:27:44.412Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10116	2024-12-06T21:27:44.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10117	2024-12-06T21:27:44.414Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10118	2024-12-06T21:27:44.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10119	2024-12-06T21:27:44.416Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10120	2024-12-06T21:27:44.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
10121	2024-12-06T21:27:44.419Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10122	2024-12-06T21:27:44.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10123	2024-12-06T21:27:44.432Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
10124	2024-12-06T21:27:44.432Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.139942ms iteration = 78 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
10125	2024-12-06T21:27:44.438Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
10126	2024-12-06T21:27:44.438Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
10127	2024-12-06T21:27:44.440Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.9315ms iteration = 78 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
10128	2024-12-06T21:27:46.542Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 388 local_addr = [fd00:1122:3344:101::c]:52742 method = GET remote_addr = [fd00:1122:3344:101::d]:63318 req_id = e65ad205-0f8f-424e-9c28-4b5ba6d5b6ac response_code = 200 uri = /e8725c8c-13ab-4916-bf16-bb7f832014d5
10129	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 27 reason = Timeout
10130	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 14 reason = Timeout
10131	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
10132	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
10133	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10134	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = external_endpoints iteration = 14 reason = Timeout
10135	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 14 reason = Timeout
10136	2024-12-06T21:27:54.407Z	INFO	nexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-06 21:17:54.407631371 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
10137	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10138	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
10139	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 16 reason = Timeout
10140	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 14 reason = Timeout
10141	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 27 reason = Timeout
10142	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
10143	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
10144	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 14 reason = Timeout
10145	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
10146	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 14 reason = Timeout
10147	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
10148	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10149	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 27 reason = Timeout
10150	2024-12-06T21:27:54.407Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
10151	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10152	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 27 reason = Timeout
10153	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 27 reason = Timeout
10154	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
10155	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
10156	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
10157	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 27 reason = Timeout
10158	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 29 reason = Timeout
10159	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
10160	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 27 reason = Timeout
10161	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10162	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
10163	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 27 reason = Timeout
10164	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
10165	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 16 reason = Timeout
10166	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 27 reason = Timeout
10167	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 14 reason = Timeout
10168	2024-12-06T21:27:54.407Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10169	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 79 reason = Timeout
10170	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
10171	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 27 reason = Timeout
10172	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
10173	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
10174	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10175	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 27 reason = Timeout
10176	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 27 reason = Timeout
10177	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 15 reason = Timeout
10178	2024-12-06T21:27:54.408Z	WARN	nexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0
10179	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 8.03\u{b5}s iteration = 15 status = {"enabled":false,"target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0"}
10180	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
10181	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 27 reason = Timeout
10182	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 79 reason = Timeout
10183	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 14 reason = Timeout
10184	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10185	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 14 reason = Timeout
10186	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10187	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
10188	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 27 reason = Timeout
10189	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
10190	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 14 reason = Timeout
10191	2024-12-06T21:27:54.408Z	INFO	nexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
10192	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
10193	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
10194	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10195	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
10196	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
10197	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
10198	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
10199	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10200	2024-12-06T21:27:54.408Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("709da000-e0da-4b29-8160-3112fd080927.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a03854a6-7a78-402b-93b4-65c6d36bf236.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a03854a6-7a78-402b-93b4-65c6d36bf236.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1589, tv_nsec: 905749896 } })
10201	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10202	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10203	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10204	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
10205	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
10206	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10207	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
10208	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
10209	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
10210	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
10211	2024-12-06T21:27:54.408Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4aff1ae3-2d71-40bf-bc46-79a7276da3c2.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4dccdb26-9cf9-40aa-9229-5d35905cf0e1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("79d475dc-44ec-44f4-9e51-ea8a753c39f3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("79d475dc-44ec-44f4-9e51-ea8a753c39f3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1589, tv_nsec: 905911508 } })
10212	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
10213	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
10214	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
10215	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
10216	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10217	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
10218	2024-12-06T21:27:54.408Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.052907ms iteration = 14 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
10219	2024-12-06T21:27:54.409Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
10220	2024-12-06T21:27:54.409Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.281491ms iteration = 14 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
10221	2024-12-06T21:27:54.409Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10222	2024-12-06T21:27:54.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10223	2024-12-06T21:27:54.410Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1589, tv_nsec: 907443324 } })
10224	2024-12-06T21:27:54.410Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
10225	2024-12-06T21:27:54.412Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 4.625136ms iteration = 27 status = {"errors":[],"records_set_to_done":[]}
10226	2024-12-06T21:27:54.412Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10227	2024-12-06T21:27:54.413Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
10228	2024-12-06T21:27:54.413Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
10229	2024-12-06T21:27:54.413Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
10230	2024-12-06T21:27:54.413Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 5.574231ms iteration = 27 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
10231	2024-12-06T21:27:54.413Z	INFO	nexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-06 21:17:54.407631371 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
10232	2024-12-06T21:27:54.413Z	DEBG	nexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 5.951558ms iteration = 14 status = {"expiration":"2024-12-06T21:17:54.407631371Z","pruned":[]}
10233	2024-12-06T21:27:54.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
10234	2024-12-06T21:27:54.414Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
10235	2024-12-06T21:27:54.414Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10236	2024-12-06T21:27:54.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10237	2024-12-06T21:27:54.414Z	DEBG	nexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 6.760361ms iteration = 14 status = {"errors":[],"found_port_ok":[]}
10238	2024-12-06T21:27:54.415Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10239	2024-12-06T21:27:54.416Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10240	2024-12-06T21:27:54.416Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10241	2024-12-06T21:27:54.417Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10242	2024-12-06T21:27:54.417Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
10243	2024-12-06T21:27:54.417Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10244	2024-12-06T21:27:54.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10245	2024-12-06T21:27:54.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
10246	2024-12-06T21:27:54.419Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10247	2024-12-06T21:27:54.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10248	2024-12-06T21:27:54.420Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
10249	2024-12-06T21:27:54.420Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10250	2024-12-06T21:27:54.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10251	2024-12-06T21:27:54.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
10252	2024-12-06T21:27:54.421Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10253	2024-12-06T21:27:54.421Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10254	2024-12-06T21:27:54.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10255	2024-12-06T21:27:54.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10256	2024-12-06T21:27:54.424Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10257	2024-12-06T21:27:54.424Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10258	2024-12-06T21:27:54.425Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
10259	2024-12-06T21:27:54.425Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
10260	2024-12-06T21:27:54.426Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1589, tv_nsec: 923498427 } })
10261	2024-12-06T21:27:54.426Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
10262	2024-12-06T21:27:54.426Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
10263	2024-12-06T21:27:54.426Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
10264	2024-12-06T21:27:54.426Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
10265	2024-12-06T21:27:54.426Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
10266	2024-12-06T21:27:54.426Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
10267	2024-12-06T21:27:54.426Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
10268	2024-12-06T21:27:54.429Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10269	2024-12-06T21:27:54.430Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10270	2024-12-06T21:27:54.430Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10271	2024-12-06T21:27:54.431Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10272	2024-12-06T21:27:54.431Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10273	2024-12-06T21:27:54.431Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10274	2024-12-06T21:27:54.431Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
10275	2024-12-06T21:27:54.431Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10276	2024-12-06T21:27:54.431Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10277	2024-12-06T21:27:54.431Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10278	2024-12-06T21:27:54.435Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10279	2024-12-06T21:27:54.435Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10280	2024-12-06T21:27:54.438Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10281	2024-12-06T21:27:54.438Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10282	2024-12-06T21:27:54.441Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
10283	2024-12-06T21:27:54.441Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10284	2024-12-06T21:27:54.441Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10285	2024-12-06T21:27:54.444Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
10286	2024-12-06T21:27:54.447Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10287	2024-12-06T21:27:54.448Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
10288	2024-12-06T21:27:54.448Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
10289	2024-12-06T21:27:54.448Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
10290	2024-12-06T21:27:54.448Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10291	2024-12-06T21:27:54.448Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10292	2024-12-06T21:27:54.449Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1639040a-f5da-4106-ac1b-42d9f99afed0, lookup_type: ById(1639040a-f5da-4106-ac1b-42d9f99afed0) } result = Ok(())
10293	2024-12-06T21:27:54.449Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10294	2024-12-06T21:27:54.450Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10295	2024-12-06T21:27:54.460Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 53.116481ms iteration = 27 status = {"errors":[],"garbage_collect_requested":[]}
10296	2024-12-06T21:27:54.461Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
10297	2024-12-06T21:27:54.462Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
10298	2024-12-06T21:27:54.466Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
10299	2024-12-06T21:27:54.466Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
10300	2024-12-06T21:27:54.470Z	DEBG	nexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
10301	2024-12-06T21:27:54.470Z	INFO	nexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
10302	2024-12-06T21:27:54.470Z	DEBG	nexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 62.144349ms iteration = 14 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
10303	2024-12-06T21:27:54.473Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10304	2024-12-06T21:27:54.473Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
10305	2024-12-06T21:27:54.474Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
10306	2024-12-06T21:27:54.474Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
10307	2024-12-06T21:27:54.474Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
10308	2024-12-06T21:27:54.474Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
10309	2024-12-06T21:27:54.474Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
10310	2024-12-06T21:27:54.476Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
10311	2024-12-06T21:27:54.479Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10312	2024-12-06T21:27:54.481Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
10313	2024-12-06T21:27:54.481Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10314	2024-12-06T21:27:54.481Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10315	2024-12-06T21:27:54.482Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 74.718796ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
10316	2024-12-06T21:27:54.483Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
10317	2024-12-06T21:27:54.483Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 75.191653ms iteration = 27 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
10318	2024-12-06T21:27:54.483Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
10319	2024-12-06T21:27:54.483Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
10320	2024-12-06T21:27:54.484Z	DEBG	nexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
10321	2024-12-06T21:27:54.484Z	INFO	nexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
10322	2024-12-06T21:27:54.484Z	DEBG	nexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 76.439304ms iteration = 14 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
10323	2024-12-06T21:27:54.486Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10324	2024-12-06T21:27:54.487Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
10325	2024-12-06T21:27:54.487Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10326	2024-12-06T21:27:54.488Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10327	2024-12-06T21:27:54.498Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
10328	2024-12-06T21:27:54.504Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
10329	2024-12-06T21:27:54.504Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
10330	2024-12-06T21:27:54.505Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "73418ac1-8709-4369-8197-51668dcec369", "content-length": "26", "date": "Fri, 06 Dec 2024 21:27:53 GMT"} })
10331	2024-12-06T21:27:54.505Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
10332	2024-12-06T21:27:54.507Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
10333	2024-12-06T21:27:54.507Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
10334	2024-12-06T21:27:54.507Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
10335	2024-12-06T21:27:54.507Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10336	2024-12-06T21:27:54.507Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
10337	2024-12-06T21:27:54.507Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10338	2024-12-06T21:27:54.532Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
10339	2024-12-06T21:27:54.534Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
10340	2024-12-06T21:27:54.534Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
10341	2024-12-06T21:27:54.544Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
10342	2024-12-06T21:27:54.544Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
10343	2024-12-06T21:27:54.544Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "af7c62bb-5c00-433e-a6e8-cbd225adb783", "content-length": "26", "date": "Fri, 06 Dec 2024 21:27:53 GMT"} })
10344	2024-12-06T21:27:54.544Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
10345	2024-12-06T21:27:54.546Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
10346	2024-12-06T21:27:54.585Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
10347	2024-12-06T21:27:54.585Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
10348	2024-12-06T21:27:54.586Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "12989077-5783-4de1-b630-a4d36b3f266f", "content-length": "372", "date": "Fri, 06 Dec 2024 21:27:53 GMT"} })
10349	2024-12-06T21:27:54.586Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10350	2024-12-06T21:27:54.586Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10351	2024-12-06T21:27:54.588Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10352	2024-12-06T21:27:54.592Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
10353	2024-12-06T21:27:54.592Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10354	2024-12-06T21:27:54.592Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10355	2024-12-06T21:27:54.595Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10356	2024-12-06T21:27:54.595Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-cbc5c1c6-5668-455a-85ef-94f4aebe0f80.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1590, tv_nsec: 92964347 } })
10357	2024-12-06T21:27:54.596Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
10358	2024-12-06T21:27:54.597Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 original_time_created = 2024-12-06 21:14:44.096531 UTC target_id = 1639040a-f5da-4106-ac1b-42d9f99afed0 time_created = 2024-12-06 21:14:44.096531 UTC
10359	2024-12-06T21:27:54.597Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 189.863665ms iteration = 79 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1639040a-f5da-4106-ac1b-42d9f99afed0","time_created":"2024-12-06T21:14:44.096531Z"}
10360	2024-12-06T21:27:54.598Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
10361	2024-12-06T21:27:54.598Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10362	2024-12-06T21:27:54.598Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10363	2024-12-06T21:27:54.611Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10364	2024-12-06T21:27:54.612Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10365	2024-12-06T21:27:54.615Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10366	2024-12-06T21:27:54.615Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10367	2024-12-06T21:27:54.616Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10368	2024-12-06T21:27:54.620Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cbc5c1c6-5668-455a-85ef-94f4aebe0f80, lookup_type: ById(cbc5c1c6-5668-455a-85ef-94f4aebe0f80) } result = Ok(())
10369	2024-12-06T21:27:54.620Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10370	2024-12-06T21:27:54.621Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10371	2024-12-06T21:27:54.623Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 215.8284ms iteration = 27 status = 0
10372	2024-12-06T21:27:54.628Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10373	2024-12-06T21:27:54.628Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10374	2024-12-06T21:27:54.631Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
10375	2024-12-06T21:27:54.634Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 226.567756ms iteration = 79 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
10376	2024-12-06T21:27:54.634Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10377	2024-12-06T21:27:54.637Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
10378	2024-12-06T21:27:54.637Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10379	2024-12-06T21:27:54.638Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10380	2024-12-06T21:27:54.646Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
10381	2024-12-06T21:27:54.646Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
10382	2024-12-06T21:27:54.648Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
10383	2024-12-06T21:27:54.648Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 05c53cbc-f66c-491f-af8d-6f291fe21596 dns_zone_name = oxide.test found_so_far = 0 version = 2
10384	2024-12-06T21:27:54.648Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10385	2024-12-06T21:27:54.648Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10386	2024-12-06T21:27:54.648Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
10387	2024-12-06T21:27:54.648Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
10388	2024-12-06T21:27:54.648Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 240.972492ms iteration = 27 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
10389	2024-12-06T21:27:54.651Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10390	2024-12-06T21:27:54.655Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
10391	2024-12-06T21:27:54.655Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10392	2024-12-06T21:27:54.656Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10393	2024-12-06T21:27:54.659Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 251.581446ms iteration = 27 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
10394	2024-12-06T21:27:54.660Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
10395	2024-12-06T21:27:54.660Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 93c3f865-bc00-42df-bef7-20e39342f84f dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
10396	2024-12-06T21:27:54.660Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10397	2024-12-06T21:27:54.660Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10398	2024-12-06T21:27:54.661Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 05c53cbc-f66c-491f-af8d-6f291fe21596 dns_zone_name = oxide.test found_so_far = 1 version = 2
10399	2024-12-06T21:27:54.661Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
10400	2024-12-06T21:27:54.661Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-06 21:15:07.045426 UTC dns_group = external generation = 2
10401	2024-12-06T21:27:54.661Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 253.61427ms iteration = 14 status = {"generation":2}
10402	2024-12-06T21:27:54.671Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
10403	2024-12-06T21:27:54.671Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
10404	2024-12-06T21:27:54.671Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "c83c7aa8-e8da-4ffb-8404-b95c70525a7d", "date": "Fri, 06 Dec 2024 21:27:53 GMT"} })
10405	2024-12-06T21:27:54.671Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10406	2024-12-06T21:27:54.671Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10407	2024-12-06T21:27:54.673Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10408	2024-12-06T21:27:54.674Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10409	2024-12-06T21:27:54.677Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10410	2024-12-06T21:27:54.677Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10411	2024-12-06T21:27:54.695Z	INFO	nexus (BackgroundTasks): VPC route manager sled cbc5c1c6-5668-455a-85ef-94f4aebe0f80 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
10412	2024-12-06T21:27:54.695Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
10413	2024-12-06T21:27:54.696Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "fce4f94c-e5f3-4181-a83e-7a4d563cfccf", "date": "Fri, 06 Dec 2024 21:27:53 GMT"} })
10414	2024-12-06T21:27:54.698Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 290.303522ms iteration = 27 status = {}
10415	2024-12-06T21:27:54.701Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
10416	2024-12-06T21:27:54.701Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "47692cd9-e578-4b3e-afda-8a924af34e43", "date": "Fri, 06 Dec 2024 21:27:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
10417	2024-12-06T21:27:54.702Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
10418	2024-12-06T21:27:54.706Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "6b75b59d-2be5-416e-8567-c0bff6bee580", "date": "Fri, 06 Dec 2024 21:27:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
10419	2024-12-06T21:27:54.706Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
10420	2024-12-06T21:27:54.708Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
10421	2024-12-06T21:27:54.708Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 300.150854ms iteration = 16 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
10422	2024-12-06T21:27:54.712Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 305.090014ms iteration = 27 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
10423	2024-12-06T21:27:54.716Z	DEBG	nexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-06T21:14:50.566199Z, time_modified: 2024-12-06T21:14:50.566199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, db_silo: Silo { identity: SiloIdentity { id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-06T21:15:06.991365Z, time_modified: 2024-12-06T21:15:06.991365Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, db_silo: Silo { identity: SiloIdentity { id: 683d02e1-6ec8-48bf-98d0-db6c94785f80, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-06T21:15:06.991365Z, time_modified: 2024-12-06T21:15:06.991365Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1" }] }) }
10424	2024-12-06T21:27:54.717Z	DEBG	nexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 309.388014ms iteration = 14 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"683d02e1-6ec8-48bf-98d0-db6c94785f80","tls_certs":["d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1"]}},"default_endpoint":{"silo_id":"683d02e1-6ec8-48bf-98d0-db6c94785f80","tls_certs":["d9bf9e2d9ac7f1d928f406ccb10ba48e42a7975122e98185d2e0bdb9c81306e1"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
10425	2024-12-06T21:27:54.718Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10426	2024-12-06T21:27:54.722Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
10427	2024-12-06T21:27:54.722Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10428	2024-12-06T21:27:54.722Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10429	2024-12-06T21:27:54.725Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 93c3f865-bc00-42df-bef7-20e39342f84f dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
10430	2024-12-06T21:27:54.725Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
10431	2024-12-06T21:27:54.725Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-06 21:15:06.025861 UTC dns_group = internal generation = 1
10432	2024-12-06T21:27:54.725Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 317.824163ms iteration = 14 status = {"generation":1}
10433	2024-12-06T21:27:54.748Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
10434	2024-12-06T21:27:54.748Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
10435	2024-12-06T21:27:54.749Z	WARN	nexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
10436	2024-12-06T21:27:54.749Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 341.45926ms iteration = 14 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
10437	2024-12-06T21:27:54.750Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10438	2024-12-06T21:27:54.751Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
10439	2024-12-06T21:27:54.751Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
10440	2024-12-06T21:27:54.752Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
10441	2024-12-06T21:27:54.753Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 346.167628ms iteration = 27 status = {}
10442	2024-12-06T21:27:54.754Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 346.536795ms iteration = 29 status = {"physical_disks_added":0}
10443	2024-12-06T21:27:54.754Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 zone_address = fd00:1122:3344:101::2
10444	2024-12-06T21:27:54.754Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
10445	2024-12-06T21:27:54.754Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe0f9d39-4c6b-4d06-9a91-b417d9a1439c", "content-length": "26", "date": "Fri, 06 Dec 2024 21:27:54 GMT"} })
10446	2024-12-06T21:27:54.754Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
10447	2024-12-06T21:27:54.756Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
10448	2024-12-06T21:27:54.765Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
10449	2024-12-06T21:27:54.765Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "3e350373-8540-4df2-9784-ff313e3ae07e", "date": "Fri, 06 Dec 2024 21:27:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
10450	2024-12-06T21:27:54.765Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
10451	2024-12-06T21:27:54.767Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "8b5f8bcd-60f9-4ef4-92fd-5db7a2a99ffd", "date": "Fri, 06 Dec 2024 21:27:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
10452	2024-12-06T21:27:54.767Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
10453	2024-12-06T21:27:54.769Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "e7bc5183-1197-43c7-84b6-03eef86d0b5a", "date": "Fri, 06 Dec 2024 21:27:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
10454	2024-12-06T21:27:54.769Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
10455	2024-12-06T21:27:54.771Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
10456	2024-12-06T21:27:54.771Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 363.881368ms iteration = 16 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
10457	2024-12-06T21:27:54.772Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
10458	2024-12-06T21:27:54.773Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "11348198-d8ce-4194-b8be-bd5d46fa0d8d", "content-length": "2", "date": "Fri, 06 Dec 2024 21:27:53 GMT"} })
10459	2024-12-06T21:27:54.774Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
10460	2024-12-06T21:27:54.774Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
10461	2024-12-06T21:27:54.776Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 368.648487ms iteration = 27 status = null
10462	2024-12-06T21:27:54.785Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10463	2024-12-06T21:27:54.787Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
10464	2024-12-06T21:27:54.787Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
10465	2024-12-06T21:27:54.787Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
10466	2024-12-06T21:27:54.850Z	DEBG	nexus (BackgroundTasks): client request SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
10467	2024-12-06T21:27:54.851Z	DEBG	nexus (BackgroundTasks): client response SledAgent = cbc5c1c6-5668-455a-85ef-94f4aebe0f80 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "34d178b1-9b31-4bff-bc34-c1b72f950aab", "content-length": "691", "date": "Fri, 06 Dec 2024 21:27:53 GMT"} })
10468	2024-12-06T21:27:54.851Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
10469	2024-12-06T21:27:54.851Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
10470	2024-12-06T21:27:54.851Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
10471	2024-12-06T21:27:54.852Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 444.339848ms iteration = 27 status = {}
10472	2024-12-06T21:27:54.891Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10473	2024-12-06T21:27:54.891Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10474	2024-12-06T21:27:54.893Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 3d56b7bf-53ed-4147-b45b-9cef154d7338, rack_id: 2d75e4a3-6363-4fd1-935e-989d31fd7d71, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(02859c9b-92d7-4b19-b425-8f266763be54) } rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
10475	2024-12-06T21:27:54.893Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10476	2024-12-06T21:27:54.894Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10477	2024-12-06T21:27:54.911Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
10478	2024-12-06T21:27:54.911Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "364aa12f-cda0-42c2-9aa4-e038e7946003", "content-length": "134", "date": "Fri, 06 Dec 2024 21:27:53 GMT"} })
10479	2024-12-06T21:27:54.911Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
10480	2024-12-06T21:27:54.911Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 02859c9b-92d7-4b19-b425-8f266763be54
10481	2024-12-06T21:27:54.911Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
10482	2024-12-06T21:27:54.911Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
10483	2024-12-06T21:27:54.911Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
10484	2024-12-06T21:27:54.911Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
10485	2024-12-06T21:27:54.911Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
10486	2024-12-06T21:27:54.911Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
10487	2024-12-06T21:27:54.915Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
10488	2024-12-06T21:27:54.915Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
10489	2024-12-06T21:27:54.916Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71
10490	2024-12-06T21:27:54.916Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10491	2024-12-06T21:27:54.916Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10492	2024-12-06T21:27:54.918Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
10493	2024-12-06T21:27:54.918Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "9c74a5bb-5754-4b17-8ea3-36b9bdd2b844", "content-length": "2", "date": "Fri, 06 Dec 2024 21:27:53 GMT"} })
10494	2024-12-06T21:27:54.918Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
10495	2024-12-06T21:27:54.919Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "5590a24f-55a6-44e8-b4ba-0caf2d470486", "content-length": "52", "date": "Fri, 06 Dec 2024 21:27:53 GMT"} })
10496	2024-12-06T21:27:54.919Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 switch_location = Switch0
10497	2024-12-06T21:27:54.919Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 2d75e4a3-6363-4fd1-935e-989d31fd7d71 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports