Max level shown: TRCE DEBG INFO WARN ERRO FATA

1	2024-12-17T21:30:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 8 reason = Timeout
2	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 9 reason = Timeout
3	2024-12-17T21:30:05.937Z	WARN	nexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5
4	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 15 reason = Timeout
5	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 23.801\u{b5}s iteration = 9 status = {"enabled":false,"target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5"}
6	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 8 reason = Timeout
7	2024-12-17T21:30:05.937Z	INFO	nexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
8	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 15 reason = Timeout
9	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
10	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 15 reason = Timeout
12	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
13	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
14	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 15 reason = Timeout
15	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 8 reason = Timeout
16	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
17	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
18	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 10 reason = Timeout
19	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
20	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 15 reason = Timeout
21	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 15 reason = Timeout
23	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 16 reason = Timeout
24	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
25	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): activating background_task = external_endpoints iteration = 8 reason = Timeout
26	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 10 reason = Timeout
27	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
28	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 8 reason = Timeout
29	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
30	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 8 reason = Timeout
31	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
32	2024-12-17T21:30:05.937Z	INFO	nexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-17 21:20:05.937172930 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
33	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 8 reason = Timeout
34	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
35	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 15 reason = Timeout
36	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
37	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 43 reason = Timeout
38	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 15 reason = Timeout
40	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
41	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 8 reason = Timeout
42	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 8 reason = Timeout
43	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
44	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
45	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 15 reason = Timeout
46	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 15 reason = Timeout
47	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
48	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
49	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 8 reason = Timeout
50	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
51	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 15 reason = Timeout
52	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
53	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
54	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 15 reason = Timeout
55	2024-12-17T21:30:05.937Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
56	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 43 reason = Timeout
58	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
59	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 15 reason = Timeout
60	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
61	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
62	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 15 reason = Timeout
63	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
64	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
65	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
66	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
67	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
68	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
69	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
70	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
71	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
72	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
73	2024-12-17T21:30:05.937Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("5dff2793-3ce8-4251-ba71-1a446966cbf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a761a47e-08b9-41c4-9bd8-8d77651dfa62.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a761a47e-08b9-41c4-9bd8-8d77651dfa62.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1664, tv_nsec: 68504449 } })
74	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
75	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
76	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
77	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
78	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
79	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
80	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
81	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
82	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
83	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
84	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
85	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
86	2024-12-17T21:30:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
87	2024-12-17T21:30:05.938Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
88	2024-12-17T21:30:05.938Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0005c310-624c-4312-b0fd-35f571746851.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6ac4d5e8-fcd9-4aa9-aed5-fb3dc293aac2.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9b804364-993d-47ba-81a1-d0ef32f96529.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("9b804364-993d-47ba-81a1-d0ef32f96529.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1664, tv_nsec: 68861289 } })
89	2024-12-17T21:30:05.938Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
90	2024-12-17T21:30:05.938Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.184262ms iteration = 8 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
91	2024-12-17T21:30:05.938Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
92	2024-12-17T21:30:05.938Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.854701ms iteration = 8 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
93	2024-12-17T21:30:05.939Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94	2024-12-17T21:30:05.940Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95	2024-12-17T21:30:05.940Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96	2024-12-17T21:30:05.940Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.645909ms iteration = 15 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
97	2024-12-17T21:30:05.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
98	2024-12-17T21:30:05.941Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
99	2024-12-17T21:30:05.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
100	2024-12-17T21:30:05.943Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
101	2024-12-17T21:30:05.943Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
102	2024-12-17T21:30:05.943Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
103	2024-12-17T21:30:05.943Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
104	2024-12-17T21:30:05.943Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
105	2024-12-17T21:30:05.943Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
106	2024-12-17T21:30:05.943Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
107	2024-12-17T21:30:05.944Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
108	2024-12-17T21:30:05.946Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1664, tv_nsec: 77503913 } })
109	2024-12-17T21:30:05.947Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
110	2024-12-17T21:30:05.947Z	INFO	nexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-17 21:20:05.937172930 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
111	2024-12-17T21:30:05.947Z	DEBG	nexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 10.402481ms iteration = 8 status = {"expiration":"2024-12-17T21:20:05.937172930Z","pruned":[]}
112	2024-12-17T21:30:05.947Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113	2024-12-17T21:30:05.951Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
114	2024-12-17T21:30:05.951Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
115	2024-12-17T21:30:05.951Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
116	2024-12-17T21:30:05.952Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
117	2024-12-17T21:30:05.952Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
118	2024-12-17T21:30:05.954Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1664, tv_nsec: 85361956 } })
119	2024-12-17T21:30:05.955Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
120	2024-12-17T21:30:05.957Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121	2024-12-17T21:30:05.959Z	DEBG	nexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 21.838141ms iteration = 8 status = {"errors":[],"found_port_ok":[]}
122	2024-12-17T21:30:05.959Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
123	2024-12-17T21:30:05.959Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
124	2024-12-17T21:30:05.959Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
125	2024-12-17T21:30:05.959Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
126	2024-12-17T21:30:05.960Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
127	2024-12-17T21:30:05.960Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
128	2024-12-17T21:30:05.960Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
129	2024-12-17T21:30:05.962Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130	2024-12-17T21:30:05.963Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
131	2024-12-17T21:30:05.963Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 25.954002ms iteration = 15 status = {"errors":[],"garbage_collect_requested":[]}
132	2024-12-17T21:30:05.963Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
133	2024-12-17T21:30:05.964Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134	2024-12-17T21:30:05.965Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
135	2024-12-17T21:30:05.965Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
136	2024-12-17T21:30:05.965Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
137	2024-12-17T21:30:05.967Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
138	2024-12-17T21:30:05.967Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
139	2024-12-17T21:30:05.967Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
140	2024-12-17T21:30:05.974Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
141	2024-12-17T21:30:05.975Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
142	2024-12-17T21:30:05.975Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143	2024-12-17T21:30:05.975Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
144	2024-12-17T21:30:05.976Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
145	2024-12-17T21:30:05.976Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146	2024-12-17T21:30:05.978Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
147	2024-12-17T21:30:05.979Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
148	2024-12-17T21:30:05.979Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
149	2024-12-17T21:30:05.980Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1664, tv_nsec: 110873066 } })
150	2024-12-17T21:30:05.980Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
151	2024-12-17T21:30:05.981Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
152	2024-12-17T21:30:05.981Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
153	2024-12-17T21:30:05.981Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
154	2024-12-17T21:30:05.981Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
155	2024-12-17T21:30:05.981Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
156	2024-12-17T21:30:05.981Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
157	2024-12-17T21:30:05.982Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
158	2024-12-17T21:30:05.982Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
159	2024-12-17T21:30:05.982Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
160	2024-12-17T21:30:05.982Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
161	2024-12-17T21:30:05.982Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
162	2024-12-17T21:30:05.982Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
163	2024-12-17T21:30:05.982Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
164	2024-12-17T21:30:05.984Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 46.620951ms iteration = 15 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
165	2024-12-17T21:30:05.986Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166	2024-12-17T21:30:05.987Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
167	2024-12-17T21:30:05.987Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168	2024-12-17T21:30:05.988Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
169	2024-12-17T21:30:05.990Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
170	2024-12-17T21:30:05.990Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 52.775288ms iteration = 15 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
171	2024-12-17T21:30:05.991Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
172	2024-12-17T21:30:05.992Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
173	2024-12-17T21:30:05.992Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
174	2024-12-17T21:30:05.992Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
175	2024-12-17T21:30:05.992Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
176	2024-12-17T21:30:05.992Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
177	2024-12-17T21:30:05.993Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
178	2024-12-17T21:30:05.993Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
179	2024-12-17T21:30:05.993Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
180	2024-12-17T21:30:05.993Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
181	2024-12-17T21:30:05.993Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
182	2024-12-17T21:30:05.994Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
183	2024-12-17T21:30:05.996Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
184	2024-12-17T21:30:05.999Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
185	2024-12-17T21:30:06.000Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
186	2024-12-17T21:30:06.006Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
187	2024-12-17T21:30:06.006Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
188	2024-12-17T21:30:06.006Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
189	2024-12-17T21:30:06.008Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
190	2024-12-17T21:30:06.012Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
191	2024-12-17T21:30:06.012Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
192	2024-12-17T21:30:06.012Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
193	2024-12-17T21:30:06.014Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
194	2024-12-17T21:30:06.015Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
195	2024-12-17T21:30:06.015Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
196	2024-12-17T21:30:06.015Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
197	2024-12-17T21:30:06.015Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
198	2024-12-17T21:30:06.015Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 78.552276ms iteration = 43 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
199	2024-12-17T21:30:06.015Z	DEBG	nexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-17T21:23:03.821125Z, time_modified: 2024-12-17T21:23:03.821125Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 81e78e1e-e55a-44f9-abdc-c9c916e1d9fd, db_silo: Silo { identity: SiloIdentity { id: 81e78e1e-e55a-44f9-abdc-c9c916e1d9fd, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-17T21:23:49.443930Z, time_modified: 2024-12-17T21:23:49.443930Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e275643b21ab02733b9616c9cd3b0193e27c7f6294b7d310d6f1726c72768e50" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 81e78e1e-e55a-44f9-abdc-c9c916e1d9fd, db_silo: Silo { identity: SiloIdentity { id: 81e78e1e-e55a-44f9-abdc-c9c916e1d9fd, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-17T21:23:49.443930Z, time_modified: 2024-12-17T21:23:49.443930Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e275643b21ab02733b9616c9cd3b0193e27c7f6294b7d310d6f1726c72768e50" }] }) }
200	2024-12-17T21:30:06.016Z	DEBG	nexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 78.869374ms iteration = 8 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"81e78e1e-e55a-44f9-abdc-c9c916e1d9fd","tls_certs":["e275643b21ab02733b9616c9cd3b0193e27c7f6294b7d310d6f1726c72768e50"]}},"default_endpoint":{"silo_id":"81e78e1e-e55a-44f9-abdc-c9c916e1d9fd","tls_certs":["e275643b21ab02733b9616c9cd3b0193e27c7f6294b7d310d6f1726c72768e50"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
201	2024-12-17T21:30:06.018Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 80.702584ms iteration = 15 status = {"errors":[],"records_set_to_done":[]}
202	2024-12-17T21:30:06.018Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
203	2024-12-17T21:30:06.019Z	DEBG	nexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
204	2024-12-17T21:30:06.019Z	INFO	nexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
205	2024-12-17T21:30:06.019Z	DEBG	nexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 82.454041ms iteration = 8 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
206	2024-12-17T21:30:06.021Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
207	2024-12-17T21:30:06.021Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
208	2024-12-17T21:30:06.021Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
209	2024-12-17T21:30:06.022Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
210	2024-12-17T21:30:06.022Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
211	2024-12-17T21:30:06.023Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
212	2024-12-17T21:30:06.023Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
213	2024-12-17T21:30:06.024Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
214	2024-12-17T21:30:06.025Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
215	2024-12-17T21:30:06.025Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 88.236848ms iteration = 43 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
216	2024-12-17T21:30:06.026Z	DEBG	nexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
217	2024-12-17T21:30:06.026Z	INFO	nexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
218	2024-12-17T21:30:06.026Z	DEBG	nexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 89.500302ms iteration = 8 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
219	2024-12-17T21:30:06.026Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 89.271225ms iteration = 15 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
220	2024-12-17T21:30:06.029Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
221	2024-12-17T21:30:06.029Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
222	2024-12-17T21:30:06.032Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
223	2024-12-17T21:30:06.032Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
224	2024-12-17T21:30:06.034Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
225	2024-12-17T21:30:06.034Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
226	2024-12-17T21:30:06.034Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
227	2024-12-17T21:30:06.034Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
228	2024-12-17T21:30:06.034Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
229	2024-12-17T21:30:06.036Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
230	2024-12-17T21:30:06.036Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = ea6acdde-77d4-4bf6-b578-d9a0a29ac37b dns_zone_name = oxide.test found_so_far = 0 version = 2
231	2024-12-17T21:30:06.036Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
232	2024-12-17T21:30:06.037Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: d3278cd4-2587-4f0f-89dd-b4d4669a6ffe, lookup_type: ById(d3278cd4-2587-4f0f-89dd-b4d4669a6ffe) } result = Ok(())
233	2024-12-17T21:30:06.037Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
234	2024-12-17T21:30:06.037Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
235	2024-12-17T21:30:06.037Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
236	2024-12-17T21:30:06.038Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
237	2024-12-17T21:30:06.040Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 102.992867ms iteration = 15 status = 0
238	2024-12-17T21:30:06.040Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
239	2024-12-17T21:30:06.040Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
240	2024-12-17T21:30:06.041Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
241	2024-12-17T21:30:06.041Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
242	2024-12-17T21:30:06.042Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
243	2024-12-17T21:30:06.043Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
244	2024-12-17T21:30:06.043Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245	2024-12-17T21:30:06.045Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
246	2024-12-17T21:30:06.045Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 574f0c35-b8f2-48fe-831c-78afc4a62aaa dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
247	2024-12-17T21:30:06.045Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
248	2024-12-17T21:30:06.045Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = ea6acdde-77d4-4bf6-b578-d9a0a29ac37b dns_zone_name = oxide.test found_so_far = 1 version = 2
249	2024-12-17T21:30:06.045Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
250	2024-12-17T21:30:06.045Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-17 21:23:49.491999 UTC dns_group = external generation = 2
251	2024-12-17T21:30:06.045Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 108.092235ms iteration = 8 status = {"generation":2}
252	2024-12-17T21:30:06.045Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
253	2024-12-17T21:30:06.045Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
254	2024-12-17T21:30:06.045Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
255	2024-12-17T21:30:06.046Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
256	2024-12-17T21:30:06.046Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
257	2024-12-17T21:30:06.046Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
258	2024-12-17T21:30:06.046Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
259	2024-12-17T21:30:06.047Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
260	2024-12-17T21:30:06.048Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
261	2024-12-17T21:30:06.050Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
262	2024-12-17T21:30:06.050Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
263	2024-12-17T21:30:06.050Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
264	2024-12-17T21:30:06.051Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
265	2024-12-17T21:30:06.051Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
266	2024-12-17T21:30:06.051Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
267	2024-12-17T21:30:06.053Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
268	2024-12-17T21:30:06.054Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
269	2024-12-17T21:30:06.054Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 574f0c35-b8f2-48fe-831c-78afc4a62aaa dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
270	2024-12-17T21:30:06.054Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
271	2024-12-17T21:30:06.054Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-17 21:23:48.446255 UTC dns_group = internal generation = 1
272	2024-12-17T21:30:06.054Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 117.243833ms iteration = 8 status = {"generation":1}
273	2024-12-17T21:30:06.067Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 130.053229ms iteration = 15 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
274	2024-12-17T21:30:06.068Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
275	2024-12-17T21:30:06.068Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
276	2024-12-17T21:30:06.068Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 130.971755ms iteration = 15 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
277	2024-12-17T21:30:06.074Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 137.042869ms iteration = 16 status = {"physical_disks_added":0}
278	2024-12-17T21:30:06.110Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
279	2024-12-17T21:30:06.134Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
280	2024-12-17T21:30:06.173Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 zone_address = fd00:1122:3344:101::2
281	2024-12-17T21:30:06.173Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
282	2024-12-17T21:30:06.173Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "fef4f5f4-1127-4e3e-9c63-736f559b0057", "content-length": "26", "date": "Tue, 17 Dec 2024 21:30:05 GMT"} })
283	2024-12-17T21:30:06.173Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
284	2024-12-17T21:30:06.175Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
285	2024-12-17T21:30:06.239Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
286	2024-12-17T21:30:06.239Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "d27aef55-928a-4006-8331-091837c02a6b", "content-length": "691", "date": "Tue, 17 Dec 2024 21:30:05 GMT"} })
287	2024-12-17T21:30:06.239Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
288	2024-12-17T21:30:06.239Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
289	2024-12-17T21:30:06.239Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
290	2024-12-17T21:30:06.241Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 304.273345ms iteration = 15 status = {}
291	2024-12-17T21:30:06.272Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
292	2024-12-17T21:30:06.272Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
293	2024-12-17T21:30:06.272Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f499f719-ace5-4fc8-8fb2-cf4df4a2c1ed", "content-length": "26", "date": "Tue, 17 Dec 2024 21:30:05 GMT"} })
294	2024-12-17T21:30:06.272Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
295	2024-12-17T21:30:06.274Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
296	2024-12-17T21:30:06.288Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
297	2024-12-17T21:30:06.288Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
298	2024-12-17T21:30:06.288Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0c7e9ece-d227-47de-b723-278f159f6bee", "content-length": "26", "date": "Tue, 17 Dec 2024 21:30:05 GMT"} })
299	2024-12-17T21:30:06.288Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
300	2024-12-17T21:30:06.289Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
301	2024-12-17T21:30:06.307Z	INFO	nexus (BackgroundTasks): VPC route manager sled d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
302	2024-12-17T21:30:06.307Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
303	2024-12-17T21:30:06.307Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "683f24c3-be0c-45b8-b965-09048ffefd2c", "content-length": "372", "date": "Tue, 17 Dec 2024 21:30:05 GMT"} })
304	2024-12-17T21:30:06.307Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
305	2024-12-17T21:30:06.307Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
306	2024-12-17T21:30:06.323Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
307	2024-12-17T21:30:06.323Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
308	2024-12-17T21:30:06.323Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "c27574bd-0546-478e-b572-4812b6309643", "date": "Tue, 17 Dec 2024 21:30:05 GMT"} })
309	2024-12-17T21:30:06.323Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
310	2024-12-17T21:30:06.324Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
311	2024-12-17T21:30:06.326Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
312	2024-12-17T21:30:06.326Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
313	2024-12-17T21:30:06.326Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
314	2024-12-17T21:30:06.326Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "9860ef40-6631-45f7-ab90-845c3108c217", "date": "Tue, 17 Dec 2024 21:30:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
315	2024-12-17T21:30:06.326Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
316	2024-12-17T21:30:06.327Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "5eecf497-3d06-4a8f-96d5-2787de6f4e01", "date": "Tue, 17 Dec 2024 21:30:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
317	2024-12-17T21:30:06.327Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
318	2024-12-17T21:30:06.329Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
319	2024-12-17T21:30:06.329Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 392.396179ms iteration = 10 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
320	2024-12-17T21:30:06.329Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
321	2024-12-17T21:30:06.330Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
322	2024-12-17T21:30:06.331Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
323	2024-12-17T21:30:06.333Z	INFO	nexus (BackgroundTasks): VPC route manager sled d3278cd4-2587-4f0f-89dd-b4d4669a6ffe push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
324	2024-12-17T21:30:06.333Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
325	2024-12-17T21:30:06.334Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "af4cfcd1-a118-482a-873d-3503c23f3323", "date": "Tue, 17 Dec 2024 21:30:05 GMT"} })
326	2024-12-17T21:30:06.335Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 398.34601ms iteration = 15 status = {}
327	2024-12-17T21:30:06.363Z	WARN	nexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
328	2024-12-17T21:30:06.363Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 425.966738ms iteration = 8 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
329	2024-12-17T21:30:06.419Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
330	2024-12-17T21:30:06.419Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "501b8154-465d-4245-ad69-130898f1c584", "content-length": "1", "date": "Tue, 17 Dec 2024 21:30:06 GMT"} })
331	2024-12-17T21:30:06.421Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
332	2024-12-17T21:30:06.422Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
333	2024-12-17T21:30:06.424Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 487.160204ms iteration = 15 status = null
334	2024-12-17T21:30:06.545Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 607.882792ms iteration = 15 status = {}
335	2024-12-17T21:30:06.551Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
336	2024-12-17T21:30:06.552Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "5c9b1925-8102-4f14-a48c-f520b26b7edf", "date": "Tue, 17 Dec 2024 21:30:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
337	2024-12-17T21:30:06.552Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
338	2024-12-17T21:30:06.553Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "4ac8b923-b1f4-4e0e-9b66-1c324d1d2aca", "date": "Tue, 17 Dec 2024 21:30:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
339	2024-12-17T21:30:06.553Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
340	2024-12-17T21:30:06.555Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "e5ecee01-5329-4654-ab9e-d2460e82f768", "date": "Tue, 17 Dec 2024 21:30:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
341	2024-12-17T21:30:06.555Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
342	2024-12-17T21:30:06.557Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
343	2024-12-17T21:30:06.557Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 620.093622ms iteration = 10 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
344	2024-12-17T21:30:06.568Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
345	2024-12-17T21:30:06.568Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
346	2024-12-17T21:30:06.570Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 1a3e1a79-c78b-4fb4-921b-c5e1e32e8b40, rack_id: 8216c8d1-8011-434d-a57a-2b1ce2e89ad8, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(d6c2617a-6b89-4779-bc70-3002fb0a7e5e) } rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0
347	2024-12-17T21:30:06.570Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
348	2024-12-17T21:30:06.570Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
349	2024-12-17T21:30:06.576Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
350	2024-12-17T21:30:06.577Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "68b7db74-62ba-4019-89a4-5ed80d2e1b4f", "content-length": "134", "date": "Tue, 17 Dec 2024 21:30:06 GMT"} })
351	2024-12-17T21:30:06.577Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
352	2024-12-17T21:30:06.577Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = d6c2617a-6b89-4779-bc70-3002fb0a7e5e
353	2024-12-17T21:30:06.577Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
354	2024-12-17T21:30:06.577Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
355	2024-12-17T21:30:06.577Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
356	2024-12-17T21:30:06.577Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
357	2024-12-17T21:30:06.577Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
358	2024-12-17T21:30:06.577Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
359	2024-12-17T21:30:06.577Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
360	2024-12-17T21:30:06.577Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
361	2024-12-17T21:30:06.577Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
362	2024-12-17T21:30:06.577Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
363	2024-12-17T21:30:06.578Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
364	2024-12-17T21:30:06.579Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
365	2024-12-17T21:30:06.579Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "c13a2a18-bffc-4362-8867-3fa37ca8ffc9", "content-length": "2", "date": "Tue, 17 Dec 2024 21:30:06 GMT"} })
366	2024-12-17T21:30:06.579Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
367	2024-12-17T21:30:06.579Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "305c4923-5b8c-4b74-94d8-c9c87435b516", "content-length": "51", "date": "Tue, 17 Dec 2024 21:30:06 GMT"} })
368	2024-12-17T21:30:06.579Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0
369	2024-12-17T21:30:06.579Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
370	2024-12-17T21:30:06.678Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "a66fbc19-9c8d-4c61-a2eb-035952a196b0", "date": "Tue, 17 Dec 2024 21:30:06 GMT"} })
371	2024-12-17T21:30:06.678Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch = Switch0
372	2024-12-17T21:30:06.678Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
373	2024-12-17T21:30:06.679Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "319a4479-d014-4609-a643-0f0caf905d8c", "content-length": "682", "date": "Tue, 17 Dec 2024 21:30:06 GMT"} })
374	2024-12-17T21:30:06.679Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
375	2024-12-17T21:30:06.679Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
376	2024-12-17T21:30:06.682Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
377	2024-12-17T21:30:06.683Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
378	2024-12-17T21:30:06.685Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
379	2024-12-17T21:30:06.685Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
380	2024-12-17T21:30:06.687Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
381	2024-12-17T21:30:06.688Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
382	2024-12-17T21:30:06.699Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 762.881787ms iteration = 15 status = {}
383	2024-12-17T21:30:10.408Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 289 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = 8e6e2456-8f9a-4690-beee-146a74829592 response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
384	2024-12-17T21:30:15.937Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 44 reason = Timeout
385	2024-12-17T21:30:15.937Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
386	2024-12-17T21:30:15.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
387	2024-12-17T21:30:15.937Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 44 reason = Timeout
388	2024-12-17T21:30:15.937Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
389	2024-12-17T21:30:15.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
390	2024-12-17T21:30:15.939Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
391	2024-12-17T21:30:15.939Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
392	2024-12-17T21:30:15.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
393	2024-12-17T21:30:15.941Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.926956ms iteration = 44 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
394	2024-12-17T21:30:15.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
395	2024-12-17T21:30:15.941Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
396	2024-12-17T21:30:15.942Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
397	2024-12-17T21:30:15.943Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
398	2024-12-17T21:30:15.943Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
399	2024-12-17T21:30:15.945Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
400	2024-12-17T21:30:15.947Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
401	2024-12-17T21:30:15.947Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
402	2024-12-17T21:30:15.947Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
403	2024-12-17T21:30:15.955Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
404	2024-12-17T21:30:15.955Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.45458ms iteration = 44 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
405	2024-12-17T21:30:20.407Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 253 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = 3f79b113-dd47-4043-a65f-b03fdfbe4e0d response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
406	2024-12-17T21:30:25.937Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 45 reason = Timeout
407	2024-12-17T21:30:25.937Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
408	2024-12-17T21:30:25.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
409	2024-12-17T21:30:25.937Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 45 reason = Timeout
410	2024-12-17T21:30:25.937Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
411	2024-12-17T21:30:25.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
412	2024-12-17T21:30:25.938Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
413	2024-12-17T21:30:25.939Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
414	2024-12-17T21:30:25.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
415	2024-12-17T21:30:25.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
416	2024-12-17T21:30:25.941Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
417	2024-12-17T21:30:25.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
418	2024-12-17T21:30:25.941Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.06356ms iteration = 45 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
419	2024-12-17T21:30:25.943Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
420	2024-12-17T21:30:25.943Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
421	2024-12-17T21:30:25.944Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
422	2024-12-17T21:30:25.947Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
423	2024-12-17T21:30:25.947Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
424	2024-12-17T21:30:25.947Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
425	2024-12-17T21:30:25.955Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
426	2024-12-17T21:30:25.955Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.189372ms iteration = 45 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
427	2024-12-17T21:30:30.407Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 275 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = dde44b5f-af42-40c3-96b7-3236c956085f response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
428	2024-12-17T21:30:35.936Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 46 reason = Timeout
429	2024-12-17T21:30:35.936Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 16 reason = Timeout
430	2024-12-17T21:30:35.936Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
431	2024-12-17T21:30:35.936Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 16 reason = Timeout
432	2024-12-17T21:30:35.936Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
433	2024-12-17T21:30:35.936Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 16 reason = Timeout
434	2024-12-17T21:30:35.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
435	2024-12-17T21:30:35.936Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
436	2024-12-17T21:30:35.936Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 17 reason = Timeout
437	2024-12-17T21:30:35.936Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 16 reason = Timeout
438	2024-12-17T21:30:35.936Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
439	2024-12-17T21:30:35.936Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
440	2024-12-17T21:30:35.936Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 16 reason = Timeout
441	2024-12-17T21:30:35.936Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 16 reason = Timeout
442	2024-12-17T21:30:35.936Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
443	2024-12-17T21:30:35.936Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 16 reason = Timeout
444	2024-12-17T21:30:35.936Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
445	2024-12-17T21:30:35.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
446	2024-12-17T21:30:35.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
447	2024-12-17T21:30:35.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
448	2024-12-17T21:30:35.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
449	2024-12-17T21:30:35.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
450	2024-12-17T21:30:35.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
451	2024-12-17T21:30:35.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
452	2024-12-17T21:30:35.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
453	2024-12-17T21:30:35.937Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 16 reason = Timeout
454	2024-12-17T21:30:35.937Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 16 reason = Timeout
455	2024-12-17T21:30:35.937Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
456	2024-12-17T21:30:35.937Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 16 reason = Timeout
457	2024-12-17T21:30:35.937Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
458	2024-12-17T21:30:35.938Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
459	2024-12-17T21:30:35.938Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 16 reason = Timeout
460	2024-12-17T21:30:35.938Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
461	2024-12-17T21:30:35.938Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 16 reason = Timeout
462	2024-12-17T21:30:35.938Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
463	2024-12-17T21:30:35.938Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 46 reason = Timeout
464	2024-12-17T21:30:35.938Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 16 reason = Timeout
465	2024-12-17T21:30:35.938Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
466	2024-12-17T21:30:35.938Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
467	2024-12-17T21:30:35.938Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 16 reason = Timeout
468	2024-12-17T21:30:35.938Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
469	2024-12-17T21:30:35.938Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
470	2024-12-17T21:30:35.938Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
471	2024-12-17T21:30:35.938Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
472	2024-12-17T21:30:35.938Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
473	2024-12-17T21:30:35.938Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
474	2024-12-17T21:30:35.938Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
475	2024-12-17T21:30:35.938Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
476	2024-12-17T21:30:35.938Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
477	2024-12-17T21:30:35.938Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
478	2024-12-17T21:30:35.938Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
479	2024-12-17T21:30:35.939Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
480	2024-12-17T21:30:35.939Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.444417ms iteration = 16 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
481	2024-12-17T21:30:35.939Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
482	2024-12-17T21:30:35.940Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
483	2024-12-17T21:30:35.942Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
484	2024-12-17T21:30:35.942Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
485	2024-12-17T21:30:35.942Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
486	2024-12-17T21:30:35.942Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
487	2024-12-17T21:30:35.942Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
488	2024-12-17T21:30:35.942Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
489	2024-12-17T21:30:35.945Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1694, tv_nsec: 75966882 } })
490	2024-12-17T21:30:35.945Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1694, tv_nsec: 76131686 } })
491	2024-12-17T21:30:35.945Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
492	2024-12-17T21:30:35.945Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
493	2024-12-17T21:30:35.945Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
494	2024-12-17T21:30:35.945Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
495	2024-12-17T21:30:35.945Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
496	2024-12-17T21:30:35.947Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
497	2024-12-17T21:30:35.949Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
498	2024-12-17T21:30:35.950Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
499	2024-12-17T21:30:35.950Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
500	2024-12-17T21:30:35.950Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
501	2024-12-17T21:30:35.950Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
502	2024-12-17T21:30:35.950Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
503	2024-12-17T21:30:35.950Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
504	2024-12-17T21:30:35.950Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
505	2024-12-17T21:30:35.951Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
506	2024-12-17T21:30:35.951Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
507	2024-12-17T21:30:35.951Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
508	2024-12-17T21:30:35.951Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
509	2024-12-17T21:30:35.952Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
510	2024-12-17T21:30:35.952Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
511	2024-12-17T21:30:35.952Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
512	2024-12-17T21:30:35.952Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 14.488702ms iteration = 16 status = {"errors":[],"garbage_collect_requested":[]}
513	2024-12-17T21:30:35.954Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 16.287111ms iteration = 16 status = {"errors":[],"records_set_to_done":[]}
514	2024-12-17T21:30:35.954Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
515	2024-12-17T21:30:35.955Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
516	2024-12-17T21:30:35.955Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
517	2024-12-17T21:30:35.955Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
518	2024-12-17T21:30:35.955Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
519	2024-12-17T21:30:35.956Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
520	2024-12-17T21:30:35.956Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
521	2024-12-17T21:30:35.956Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
522	2024-12-17T21:30:35.956Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1694, tv_nsec: 87618257 } })
523	2024-12-17T21:30:35.957Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
524	2024-12-17T21:30:35.957Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
525	2024-12-17T21:30:35.957Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
526	2024-12-17T21:30:35.958Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
527	2024-12-17T21:30:35.958Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
528	2024-12-17T21:30:35.958Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 20.793233ms iteration = 16 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
529	2024-12-17T21:30:35.960Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
530	2024-12-17T21:30:35.960Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
531	2024-12-17T21:30:35.961Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
532	2024-12-17T21:30:35.963Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
533	2024-12-17T21:30:35.963Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
534	2024-12-17T21:30:35.963Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
535	2024-12-17T21:30:35.963Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
536	2024-12-17T21:30:35.963Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 25.902931ms iteration = 16 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
537	2024-12-17T21:30:35.964Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
538	2024-12-17T21:30:35.964Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
539	2024-12-17T21:30:35.964Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
540	2024-12-17T21:30:35.964Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
541	2024-12-17T21:30:35.964Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
542	2024-12-17T21:30:35.964Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
543	2024-12-17T21:30:35.977Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
544	2024-12-17T21:30:35.977Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 40.64888ms iteration = 46 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
545	2024-12-17T21:30:35.981Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
546	2024-12-17T21:30:35.981Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
547	2024-12-17T21:30:35.981Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
548	2024-12-17T21:30:35.981Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
549	2024-12-17T21:30:35.981Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
550	2024-12-17T21:30:35.981Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
551	2024-12-17T21:30:35.982Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
552	2024-12-17T21:30:35.982Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
553	2024-12-17T21:30:35.983Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 46.739825ms iteration = 16 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
554	2024-12-17T21:30:35.983Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
555	2024-12-17T21:30:35.983Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
556	2024-12-17T21:30:35.983Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
557	2024-12-17T21:30:35.983Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
558	2024-12-17T21:30:35.989Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 51.285157ms iteration = 46 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
559	2024-12-17T21:30:35.994Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 56.957782ms iteration = 16 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
560	2024-12-17T21:30:35.997Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 60.300202ms iteration = 17 status = {"physical_disks_added":0}
561	2024-12-17T21:30:36.042Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
562	2024-12-17T21:30:36.042Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
563	2024-12-17T21:30:36.042Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ae81731-ce65-4b93-8286-eb5d8e23c440", "content-length": "26", "date": "Tue, 17 Dec 2024 21:30:36 GMT"} })
564	2024-12-17T21:30:36.042Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
565	2024-12-17T21:30:36.044Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
566	2024-12-17T21:30:36.064Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
567	2024-12-17T21:30:36.065Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
568	2024-12-17T21:30:36.078Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 zone_address = fd00:1122:3344:101::2
569	2024-12-17T21:30:36.078Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
570	2024-12-17T21:30:36.078Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "107a0695-dd5a-456e-885d-f99946f32741", "content-length": "26", "date": "Tue, 17 Dec 2024 21:30:36 GMT"} })
571	2024-12-17T21:30:36.078Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
572	2024-12-17T21:30:36.080Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
573	2024-12-17T21:30:36.109Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
574	2024-12-17T21:30:36.109Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
575	2024-12-17T21:30:36.109Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b0afb16-f2b1-4ebf-accb-177fde665f81", "content-length": "26", "date": "Tue, 17 Dec 2024 21:30:36 GMT"} })
576	2024-12-17T21:30:36.109Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
577	2024-12-17T21:30:36.111Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
578	2024-12-17T21:30:36.160Z	INFO	nexus (BackgroundTasks): VPC route manager sled d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
579	2024-12-17T21:30:36.160Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
580	2024-12-17T21:30:36.162Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
581	2024-12-17T21:30:36.162Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
582	2024-12-17T21:30:36.162Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
583	2024-12-17T21:30:36.164Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
584	2024-12-17T21:30:36.164Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
585	2024-12-17T21:30:36.164Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
586	2024-12-17T21:30:36.164Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
587	2024-12-17T21:30:36.166Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
588	2024-12-17T21:30:36.167Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: d3278cd4-2587-4f0f-89dd-b4d4669a6ffe, lookup_type: ById(d3278cd4-2587-4f0f-89dd-b4d4669a6ffe) } result = Ok(())
589	2024-12-17T21:30:36.167Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
590	2024-12-17T21:30:36.167Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
591	2024-12-17T21:30:36.167Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
592	2024-12-17T21:30:36.167Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "240832ae-2c20-44a1-a628-dbf353b8eb97", "content-length": "1", "date": "Tue, 17 Dec 2024 21:30:35 GMT"} })
593	2024-12-17T21:30:36.168Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
594	2024-12-17T21:30:36.168Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
595	2024-12-17T21:30:36.168Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
596	2024-12-17T21:30:36.169Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
597	2024-12-17T21:30:36.169Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 232.76905ms iteration = 16 status = 0
598	2024-12-17T21:30:36.169Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
599	2024-12-17T21:30:36.171Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 234.912288ms iteration = 16 status = null
600	2024-12-17T21:30:36.199Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
601	2024-12-17T21:30:36.199Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
602	2024-12-17T21:30:36.199Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 261.911128ms iteration = 16 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
603	2024-12-17T21:30:36.208Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 271.901639ms iteration = 16 status = {}
604	2024-12-17T21:30:36.232Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
605	2024-12-17T21:30:36.238Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "f2d16802-dfc8-47d8-af4c-3227180c2269", "content-length": "691", "date": "Tue, 17 Dec 2024 21:30:35 GMT"} })
606	2024-12-17T21:30:36.238Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
607	2024-12-17T21:30:36.238Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
608	2024-12-17T21:30:36.238Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
609	2024-12-17T21:30:36.238Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6ce0555-8c64-4ce5-b34f-44f0e4681249", "content-length": "372", "date": "Tue, 17 Dec 2024 21:30:36 GMT"} })
610	2024-12-17T21:30:36.238Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
611	2024-12-17T21:30:36.238Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
612	2024-12-17T21:30:36.238Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 302.176978ms iteration = 16 status = {}
613	2024-12-17T21:30:36.274Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
614	2024-12-17T21:30:36.274Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
615	2024-12-17T21:30:36.276Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 1a3e1a79-c78b-4fb4-921b-c5e1e32e8b40, rack_id: 8216c8d1-8011-434d-a57a-2b1ce2e89ad8, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(d6c2617a-6b89-4779-bc70-3002fb0a7e5e) } rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0
616	2024-12-17T21:30:36.276Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
617	2024-12-17T21:30:36.276Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
618	2024-12-17T21:30:36.286Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
619	2024-12-17T21:30:36.286Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d6d5bb8-d724-4898-b921-8f2e4bdc869e", "content-length": "134", "date": "Tue, 17 Dec 2024 21:30:35 GMT"} })
620	2024-12-17T21:30:36.286Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
621	2024-12-17T21:30:36.286Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = d6c2617a-6b89-4779-bc70-3002fb0a7e5e
622	2024-12-17T21:30:36.286Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
623	2024-12-17T21:30:36.286Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
624	2024-12-17T21:30:36.286Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
625	2024-12-17T21:30:36.286Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
626	2024-12-17T21:30:36.286Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
627	2024-12-17T21:30:36.286Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
628	2024-12-17T21:30:36.286Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
629	2024-12-17T21:30:36.286Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
630	2024-12-17T21:30:36.287Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
631	2024-12-17T21:30:36.287Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
632	2024-12-17T21:30:36.287Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
633	2024-12-17T21:30:36.287Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
634	2024-12-17T21:30:36.287Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
635	2024-12-17T21:30:36.287Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "aef6ee78-0c8b-4f00-a469-c5289d4a05d2", "date": "Tue, 17 Dec 2024 21:30:36 GMT"} })
636	2024-12-17T21:30:36.287Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
637	2024-12-17T21:30:36.287Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
638	2024-12-17T21:30:36.289Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
639	2024-12-17T21:30:36.289Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc7cc69d-ef2d-4425-9062-7f5e4d0104de", "content-length": "2", "date": "Tue, 17 Dec 2024 21:30:35 GMT"} })
640	2024-12-17T21:30:36.289Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
641	2024-12-17T21:30:36.289Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "92794edf-ad71-4352-a613-ab3213ad25be", "content-length": "51", "date": "Tue, 17 Dec 2024 21:30:35 GMT"} })
642	2024-12-17T21:30:36.289Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0
643	2024-12-17T21:30:36.289Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
644	2024-12-17T21:30:36.289Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
645	2024-12-17T21:30:36.290Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
646	2024-12-17T21:30:36.293Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
647	2024-12-17T21:30:36.293Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
648	2024-12-17T21:30:36.297Z	INFO	nexus (BackgroundTasks): VPC route manager sled d3278cd4-2587-4f0f-89dd-b4d4669a6ffe push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
649	2024-12-17T21:30:36.297Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
650	2024-12-17T21:30:36.385Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "6a08d7b9-63be-4fb5-9cff-cc4c85b0e1f8", "date": "Tue, 17 Dec 2024 21:30:36 GMT"} })
651	2024-12-17T21:30:36.385Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch = Switch0
652	2024-12-17T21:30:36.385Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
653	2024-12-17T21:30:36.385Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "231bd51d-a927-48b5-b0dd-42f8e5c9c853", "content-length": "682", "date": "Tue, 17 Dec 2024 21:30:36 GMT"} })
654	2024-12-17T21:30:36.385Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
655	2024-12-17T21:30:36.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
656	2024-12-17T21:30:36.386Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "af17641e-618e-44ba-8297-3228527b58e6", "date": "Tue, 17 Dec 2024 21:30:36 GMT"} })
657	2024-12-17T21:30:36.387Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 449.277729ms iteration = 16 status = {}
658	2024-12-17T21:30:36.388Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
659	2024-12-17T21:30:36.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
660	2024-12-17T21:30:36.392Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
661	2024-12-17T21:30:36.393Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
662	2024-12-17T21:30:36.395Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
663	2024-12-17T21:30:36.395Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
664	2024-12-17T21:30:36.401Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 465.134818ms iteration = 16 status = {}
665	2024-12-17T21:30:40.408Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 251 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = fa9ed284-e728-4deb-a4c8-541de2f52b95 response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
666	2024-12-17T21:30:45.936Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 47 reason = Timeout
667	2024-12-17T21:30:45.936Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
668	2024-12-17T21:30:45.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
669	2024-12-17T21:30:45.937Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 47 reason = Timeout
670	2024-12-17T21:30:45.937Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
671	2024-12-17T21:30:45.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
672	2024-12-17T21:30:45.938Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
673	2024-12-17T21:30:45.939Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
674	2024-12-17T21:30:45.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
675	2024-12-17T21:30:45.940Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
676	2024-12-17T21:30:45.940Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
677	2024-12-17T21:30:45.940Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
678	2024-12-17T21:30:45.942Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
679	2024-12-17T21:30:45.942Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
680	2024-12-17T21:30:45.944Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
681	2024-12-17T21:30:45.947Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
682	2024-12-17T21:30:45.947Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
683	2024-12-17T21:30:45.947Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
684	2024-12-17T21:30:45.960Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
685	2024-12-17T21:30:45.960Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.906587ms iteration = 47 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
686	2024-12-17T21:30:45.974Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 37.449793ms iteration = 47 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
687	2024-12-17T21:30:50.408Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 302 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = 5d241cfa-f011-4be7-aca8-d16663a571d2 response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
688	2024-12-17T21:30:55.936Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 48 reason = Timeout
689	2024-12-17T21:30:55.936Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
690	2024-12-17T21:30:55.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
691	2024-12-17T21:30:55.937Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 48 reason = Timeout
692	2024-12-17T21:30:55.937Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
693	2024-12-17T21:30:55.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
694	2024-12-17T21:30:55.938Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
695	2024-12-17T21:30:55.939Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
696	2024-12-17T21:30:55.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
697	2024-12-17T21:30:55.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
698	2024-12-17T21:30:55.941Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
699	2024-12-17T21:30:55.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
700	2024-12-17T21:30:55.941Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.122721ms iteration = 48 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
701	2024-12-17T21:30:55.942Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
702	2024-12-17T21:30:55.942Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
703	2024-12-17T21:30:55.944Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
704	2024-12-17T21:30:55.946Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
705	2024-12-17T21:30:55.946Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
706	2024-12-17T21:30:55.946Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
707	2024-12-17T21:30:55.959Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
708	2024-12-17T21:30:55.959Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.125736ms iteration = 48 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
709	2024-12-17T21:31:00.407Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 261 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = 78fda167-1fc5-439d-8c62-89117daafecc response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
710	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 9 reason = Timeout
711	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 10 reason = Timeout
712	2024-12-17T21:31:05.936Z	WARN	nexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5
713	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 13.69\u{b5}s iteration = 10 status = {"enabled":false,"target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5"}
714	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 17 reason = Timeout
715	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 17 reason = Timeout
716	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 11 reason = Timeout
717	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
718	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
719	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 17 reason = Timeout
720	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 9 reason = Timeout
721	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
722	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
723	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
724	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 9 reason = Timeout
725	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 17 reason = Timeout
726	2024-12-17T21:31:05.936Z	INFO	nexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
727	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
728	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 18 reason = Timeout
729	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 9 reason = Timeout
730	2024-12-17T21:31:05.936Z	INFO	nexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-17 21:21:05.936235638 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
731	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
732	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 17 reason = Timeout
733	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
734	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
735	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 17 reason = Timeout
736	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
737	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 11 reason = Timeout
738	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 9 reason = Timeout
739	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = external_endpoints iteration = 9 reason = Timeout
740	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 17 reason = Timeout
741	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
742	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
743	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 49 reason = Timeout
744	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 9 reason = Timeout
745	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
746	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
747	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
748	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
749	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
750	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
751	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
752	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
753	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
754	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 17 reason = Timeout
755	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
756	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 9 reason = Timeout
757	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 9 reason = Timeout
758	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
759	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
760	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 17 reason = Timeout
761	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
762	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
763	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 17 reason = Timeout
764	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
765	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
766	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 9 reason = Timeout
767	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
768	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 17 reason = Timeout
769	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
770	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
771	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
772	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 17 reason = Timeout
773	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 17 reason = Timeout
774	2024-12-17T21:31:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
775	2024-12-17T21:31:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
776	2024-12-17T21:31:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
777	2024-12-17T21:31:05.937Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
778	2024-12-17T21:31:05.937Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
779	2024-12-17T21:31:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
780	2024-12-17T21:31:05.937Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 17 reason = Timeout
781	2024-12-17T21:31:05.937Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 49 reason = Timeout
782	2024-12-17T21:31:05.937Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
783	2024-12-17T21:31:05.937Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
784	2024-12-17T21:31:05.937Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
785	2024-12-17T21:31:05.937Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0005c310-624c-4312-b0fd-35f571746851.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6ac4d5e8-fcd9-4aa9-aed5-fb3dc293aac2.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9b804364-993d-47ba-81a1-d0ef32f96529.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("9b804364-993d-47ba-81a1-d0ef32f96529.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1724, tv_nsec: 68185071 } })
786	2024-12-17T21:31:05.937Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("5dff2793-3ce8-4251-ba71-1a446966cbf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a761a47e-08b9-41c4-9bd8-8d77651dfa62.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a761a47e-08b9-41c4-9bd8-8d77651dfa62.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1724, tv_nsec: 68186421 } })
787	2024-12-17T21:31:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
788	2024-12-17T21:31:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
789	2024-12-17T21:31:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
790	2024-12-17T21:31:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
791	2024-12-17T21:31:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
792	2024-12-17T21:31:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
793	2024-12-17T21:31:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
794	2024-12-17T21:31:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
795	2024-12-17T21:31:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
796	2024-12-17T21:31:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
797	2024-12-17T21:31:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
798	2024-12-17T21:31:05.937Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
799	2024-12-17T21:31:05.937Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.11779ms iteration = 9 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
800	2024-12-17T21:31:05.937Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
801	2024-12-17T21:31:05.937Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
802	2024-12-17T21:31:05.937Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.507791ms iteration = 9 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
803	2024-12-17T21:31:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
804	2024-12-17T21:31:05.939Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
805	2024-12-17T21:31:05.939Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.521145ms iteration = 17 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
806	2024-12-17T21:31:05.939Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
807	2024-12-17T21:31:05.940Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1724, tv_nsec: 72045355 } })
808	2024-12-17T21:31:05.941Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
809	2024-12-17T21:31:05.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
810	2024-12-17T21:31:05.941Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
811	2024-12-17T21:31:05.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
812	2024-12-17T21:31:05.941Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
813	2024-12-17T21:31:05.941Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
814	2024-12-17T21:31:05.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
815	2024-12-17T21:31:05.942Z	INFO	nexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-17 21:21:05.936235638 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
816	2024-12-17T21:31:05.942Z	DEBG	nexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 5.967202ms iteration = 9 status = {"expiration":"2024-12-17T21:21:05.936235638Z","pruned":[]}
817	2024-12-17T21:31:05.942Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
818	2024-12-17T21:31:05.943Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
819	2024-12-17T21:31:05.943Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
820	2024-12-17T21:31:05.943Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
821	2024-12-17T21:31:05.944Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1724, tv_nsec: 76031783 } })
822	2024-12-17T21:31:05.944Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
823	2024-12-17T21:31:05.945Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
824	2024-12-17T21:31:05.946Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
825	2024-12-17T21:31:05.946Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
826	2024-12-17T21:31:05.946Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
827	2024-12-17T21:31:05.946Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
828	2024-12-17T21:31:05.946Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
829	2024-12-17T21:31:05.947Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
830	2024-12-17T21:31:05.947Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
831	2024-12-17T21:31:05.947Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
832	2024-12-17T21:31:05.947Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
833	2024-12-17T21:31:05.948Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
834	2024-12-17T21:31:05.950Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
835	2024-12-17T21:31:05.950Z	DEBG	nexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 13.978608ms iteration = 9 status = {"errors":[],"found_port_ok":[]}
836	2024-12-17T21:31:05.951Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
837	2024-12-17T21:31:05.951Z	DEBG	nexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
838	2024-12-17T21:31:05.951Z	INFO	nexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
839	2024-12-17T21:31:05.951Z	DEBG	nexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 14.380079ms iteration = 9 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
840	2024-12-17T21:31:05.951Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
841	2024-12-17T21:31:05.951Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
842	2024-12-17T21:31:05.951Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
843	2024-12-17T21:31:05.952Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
844	2024-12-17T21:31:05.952Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
845	2024-12-17T21:31:05.954Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
846	2024-12-17T21:31:05.955Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
847	2024-12-17T21:31:05.956Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
848	2024-12-17T21:31:05.956Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
849	2024-12-17T21:31:05.956Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
850	2024-12-17T21:31:05.956Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
851	2024-12-17T21:31:05.959Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
852	2024-12-17T21:31:05.959Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
853	2024-12-17T21:31:05.959Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1724, tv_nsec: 90945657 } })
854	2024-12-17T21:31:05.959Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
855	2024-12-17T21:31:05.959Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
856	2024-12-17T21:31:05.960Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
857	2024-12-17T21:31:05.960Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
858	2024-12-17T21:31:05.960Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
859	2024-12-17T21:31:05.995Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
860	2024-12-17T21:31:05.995Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
861	2024-12-17T21:31:05.995Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
862	2024-12-17T21:31:05.995Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
863	2024-12-17T21:31:05.995Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
864	2024-12-17T21:31:05.995Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
865	2024-12-17T21:31:05.995Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
866	2024-12-17T21:31:05.997Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
867	2024-12-17T21:31:05.997Z	DEBG	nexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
868	2024-12-17T21:31:05.997Z	INFO	nexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
869	2024-12-17T21:31:05.997Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
870	2024-12-17T21:31:05.997Z	DEBG	nexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 61.445653ms iteration = 9 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
871	2024-12-17T21:31:05.997Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
872	2024-12-17T21:31:05.997Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
873	2024-12-17T21:31:05.997Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
874	2024-12-17T21:31:05.998Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 61.560166ms iteration = 17 status = {"errors":[],"garbage_collect_requested":[]}
875	2024-12-17T21:31:05.998Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
876	2024-12-17T21:31:06.000Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 63.433057ms iteration = 49 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
877	2024-12-17T21:31:06.000Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
878	2024-12-17T21:31:06.000Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
879	2024-12-17T21:31:06.000Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
880	2024-12-17T21:31:06.001Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
881	2024-12-17T21:31:06.001Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
882	2024-12-17T21:31:06.001Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
883	2024-12-17T21:31:06.001Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
884	2024-12-17T21:31:06.003Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
885	2024-12-17T21:31:06.003Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
886	2024-12-17T21:31:06.006Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
887	2024-12-17T21:31:06.009Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 72.658096ms iteration = 17 status = {"errors":[],"records_set_to_done":[]}
888	2024-12-17T21:31:06.010Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
889	2024-12-17T21:31:06.013Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
890	2024-12-17T21:31:06.014Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
891	2024-12-17T21:31:06.015Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
892	2024-12-17T21:31:06.017Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
893	2024-12-17T21:31:06.017Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
894	2024-12-17T21:31:06.018Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
895	2024-12-17T21:31:06.023Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
896	2024-12-17T21:31:06.023Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 87.024565ms iteration = 17 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
897	2024-12-17T21:31:06.024Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
898	2024-12-17T21:31:06.024Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 88.174956ms iteration = 49 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
899	2024-12-17T21:31:06.025Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
900	2024-12-17T21:31:06.027Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
901	2024-12-17T21:31:06.027Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
902	2024-12-17T21:31:06.027Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
903	2024-12-17T21:31:06.028Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
904	2024-12-17T21:31:06.028Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
905	2024-12-17T21:31:06.031Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
906	2024-12-17T21:31:06.031Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
907	2024-12-17T21:31:06.037Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
908	2024-12-17T21:31:06.037Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
909	2024-12-17T21:31:06.037Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
910	2024-12-17T21:31:06.037Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
911	2024-12-17T21:31:06.037Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
912	2024-12-17T21:31:06.038Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
913	2024-12-17T21:31:06.040Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 103.700736ms iteration = 17 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
914	2024-12-17T21:31:06.045Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
915	2024-12-17T21:31:06.045Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
916	2024-12-17T21:31:06.047Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
917	2024-12-17T21:31:06.047Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
918	2024-12-17T21:31:06.047Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
919	2024-12-17T21:31:06.047Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
920	2024-12-17T21:31:06.048Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
921	2024-12-17T21:31:06.049Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 113.034639ms iteration = 17 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
922	2024-12-17T21:31:06.049Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
923	2024-12-17T21:31:06.053Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
924	2024-12-17T21:31:06.053Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
925	2024-12-17T21:31:06.053Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
926	2024-12-17T21:31:06.056Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
927	2024-12-17T21:31:06.057Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 120.350897ms iteration = 17 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
928	2024-12-17T21:31:06.057Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
929	2024-12-17T21:31:06.057Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
930	2024-12-17T21:31:06.057Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
931	2024-12-17T21:31:06.057Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
932	2024-12-17T21:31:06.057Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
933	2024-12-17T21:31:06.057Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
934	2024-12-17T21:31:06.058Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
935	2024-12-17T21:31:06.059Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
936	2024-12-17T21:31:06.060Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
937	2024-12-17T21:31:06.060Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
938	2024-12-17T21:31:06.060Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
939	2024-12-17T21:31:06.062Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
940	2024-12-17T21:31:06.062Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
941	2024-12-17T21:31:06.062Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
942	2024-12-17T21:31:06.064Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
943	2024-12-17T21:31:06.064Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = ea6acdde-77d4-4bf6-b578-d9a0a29ac37b dns_zone_name = oxide.test found_so_far = 0 version = 2
944	2024-12-17T21:31:06.064Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
945	2024-12-17T21:31:06.064Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
946	2024-12-17T21:31:06.066Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
947	2024-12-17T21:31:06.070Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
948	2024-12-17T21:31:06.070Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
949	2024-12-17T21:31:06.070Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
950	2024-12-17T21:31:06.072Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = ea6acdde-77d4-4bf6-b578-d9a0a29ac37b dns_zone_name = oxide.test found_so_far = 1 version = 2
951	2024-12-17T21:31:06.072Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
952	2024-12-17T21:31:06.072Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-17 21:23:49.491999 UTC dns_group = external generation = 2
953	2024-12-17T21:31:06.072Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 136.299089ms iteration = 9 status = {"generation":2}
954	2024-12-17T21:31:06.078Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
955	2024-12-17T21:31:06.078Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
956	2024-12-17T21:31:06.078Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 141.348465ms iteration = 17 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
957	2024-12-17T21:31:06.079Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
958	2024-12-17T21:31:06.081Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: d3278cd4-2587-4f0f-89dd-b4d4669a6ffe, lookup_type: ById(d3278cd4-2587-4f0f-89dd-b4d4669a6ffe) } result = Ok(())
959	2024-12-17T21:31:06.081Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
960	2024-12-17T21:31:06.082Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
961	2024-12-17T21:31:06.084Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 148.353925ms iteration = 17 status = 0
962	2024-12-17T21:31:06.091Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
963	2024-12-17T21:31:06.091Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 574f0c35-b8f2-48fe-831c-78afc4a62aaa dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
964	2024-12-17T21:31:06.091Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
965	2024-12-17T21:31:06.091Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
966	2024-12-17T21:31:06.093Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
967	2024-12-17T21:31:06.096Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
968	2024-12-17T21:31:06.096Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
969	2024-12-17T21:31:06.097Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
970	2024-12-17T21:31:06.099Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 574f0c35-b8f2-48fe-831c-78afc4a62aaa dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
971	2024-12-17T21:31:06.099Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
972	2024-12-17T21:31:06.099Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-17 21:23:48.446255 UTC dns_group = internal generation = 1
973	2024-12-17T21:31:06.099Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 163.31274ms iteration = 9 status = {"generation":1}
974	2024-12-17T21:31:06.102Z	DEBG	nexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-17T21:23:03.821125Z, time_modified: 2024-12-17T21:23:03.821125Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 81e78e1e-e55a-44f9-abdc-c9c916e1d9fd, db_silo: Silo { identity: SiloIdentity { id: 81e78e1e-e55a-44f9-abdc-c9c916e1d9fd, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-17T21:23:49.443930Z, time_modified: 2024-12-17T21:23:49.443930Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e275643b21ab02733b9616c9cd3b0193e27c7f6294b7d310d6f1726c72768e50" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 81e78e1e-e55a-44f9-abdc-c9c916e1d9fd, db_silo: Silo { identity: SiloIdentity { id: 81e78e1e-e55a-44f9-abdc-c9c916e1d9fd, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-17T21:23:49.443930Z, time_modified: 2024-12-17T21:23:49.443930Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e275643b21ab02733b9616c9cd3b0193e27c7f6294b7d310d6f1726c72768e50" }] }) }
975	2024-12-17T21:31:06.102Z	DEBG	nexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 166.471496ms iteration = 9 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"81e78e1e-e55a-44f9-abdc-c9c916e1d9fd","tls_certs":["e275643b21ab02733b9616c9cd3b0193e27c7f6294b7d310d6f1726c72768e50"]}},"default_endpoint":{"silo_id":"81e78e1e-e55a-44f9-abdc-c9c916e1d9fd","tls_certs":["e275643b21ab02733b9616c9cd3b0193e27c7f6294b7d310d6f1726c72768e50"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
976	2024-12-17T21:31:06.103Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
977	2024-12-17T21:31:06.103Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
978	2024-12-17T21:31:06.105Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
979	2024-12-17T21:31:06.106Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
980	2024-12-17T21:31:06.107Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
981	2024-12-17T21:31:06.107Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
982	2024-12-17T21:31:06.108Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
983	2024-12-17T21:31:06.110Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
984	2024-12-17T21:31:06.110Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
985	2024-12-17T21:31:06.111Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
986	2024-12-17T21:31:06.122Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
987	2024-12-17T21:31:06.122Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
988	2024-12-17T21:31:06.122Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "90aae277-1acb-4b48-99f2-04969076c794", "content-length": "26", "date": "Tue, 17 Dec 2024 21:31:05 GMT"} })
989	2024-12-17T21:31:06.122Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
990	2024-12-17T21:31:06.124Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
991	2024-12-17T21:31:06.135Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 198.978015ms iteration = 18 status = {"physical_disks_added":0}
992	2024-12-17T21:31:06.155Z	WARN	nexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
993	2024-12-17T21:31:06.155Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 218.594286ms iteration = 9 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
994	2024-12-17T21:31:06.217Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
995	2024-12-17T21:31:06.246Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
996	2024-12-17T21:31:06.304Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
997	2024-12-17T21:31:06.305Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "1911157c-9bbc-46a0-bad3-57290c9d8989", "content-length": "1", "date": "Tue, 17 Dec 2024 21:31:05 GMT"} })
998	2024-12-17T21:31:06.306Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
999	2024-12-17T21:31:06.306Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1000	2024-12-17T21:31:06.309Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 373.024685ms iteration = 17 status = null
1001	2024-12-17T21:31:06.313Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
1002	2024-12-17T21:31:06.313Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "a5dcfa89-7b08-41a5-801f-1bd8a433bf17", "content-length": "691", "date": "Tue, 17 Dec 2024 21:31:06 GMT"} })
1003	2024-12-17T21:31:06.313Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
1004	2024-12-17T21:31:06.313Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
1005	2024-12-17T21:31:06.313Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
1006	2024-12-17T21:31:06.315Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 378.984047ms iteration = 17 status = {}
1007	2024-12-17T21:31:06.317Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 zone_address = fd00:1122:3344:101::2
1008	2024-12-17T21:31:06.317Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1009	2024-12-17T21:31:06.318Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c7a8689-51fd-4edd-aefe-2b1fef1b16e0", "content-length": "26", "date": "Tue, 17 Dec 2024 21:31:05 GMT"} })
1010	2024-12-17T21:31:06.318Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1011	2024-12-17T21:31:06.319Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
1012	2024-12-17T21:31:06.319Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
1013	2024-12-17T21:31:06.319Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1014	2024-12-17T21:31:06.320Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "15dd12be-5887-4274-9478-e48110b2798a", "content-length": "26", "date": "Tue, 17 Dec 2024 21:31:05 GMT"} })
1015	2024-12-17T21:31:06.320Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1016	2024-12-17T21:31:06.322Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
1017	2024-12-17T21:31:06.325Z	INFO	nexus (BackgroundTasks): VPC route manager sled d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
1018	2024-12-17T21:31:06.325Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
1019	2024-12-17T21:31:06.406Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
1020	2024-12-17T21:31:06.407Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "27845fb0-1e11-4220-b803-4c0f6e0c0740", "date": "Tue, 17 Dec 2024 21:31:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
1021	2024-12-17T21:31:06.407Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
1022	2024-12-17T21:31:06.408Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
1023	2024-12-17T21:31:06.409Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "829bfcd9-2072-4a98-9aef-7e53ba2f1ede", "date": "Tue, 17 Dec 2024 21:31:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
1024	2024-12-17T21:31:06.409Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
1025	2024-12-17T21:31:06.410Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
1026	2024-12-17T21:31:06.410Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 474.296086ms iteration = 11 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
1027	2024-12-17T21:31:06.421Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "15ed38b7-0d66-4955-b309-6e9f64fdf89f", "content-length": "372", "date": "Tue, 17 Dec 2024 21:31:06 GMT"} })
1028	2024-12-17T21:31:06.421Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1029	2024-12-17T21:31:06.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1030	2024-12-17T21:31:06.432Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 496.61916ms iteration = 17 status = {}
1031	2024-12-17T21:31:06.463Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
1032	2024-12-17T21:31:06.463Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
1033	2024-12-17T21:31:06.463Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "ca2abef5-b2f8-497a-ae03-946c19bd35ea", "date": "Tue, 17 Dec 2024 21:31:06 GMT"} })
1034	2024-12-17T21:31:06.463Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1035	2024-12-17T21:31:06.463Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1036	2024-12-17T21:31:06.465Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1037	2024-12-17T21:31:06.466Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1038	2024-12-17T21:31:06.469Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1039	2024-12-17T21:31:06.469Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1040	2024-12-17T21:31:06.474Z	INFO	nexus (BackgroundTasks): VPC route manager sled d3278cd4-2587-4f0f-89dd-b4d4669a6ffe push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
1041	2024-12-17T21:31:06.474Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
1042	2024-12-17T21:31:06.474Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "0faeb8bb-6f68-4395-861c-89d73dd36aeb", "date": "Tue, 17 Dec 2024 21:31:06 GMT"} })
1043	2024-12-17T21:31:06.475Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 538.957486ms iteration = 17 status = {}
1044	2024-12-17T21:31:06.480Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
1045	2024-12-17T21:31:06.481Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "27ab892c-da1c-42eb-a0fa-8c0722499109", "date": "Tue, 17 Dec 2024 21:31:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
1046	2024-12-17T21:31:06.481Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
1047	2024-12-17T21:31:06.483Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "6dde8735-520e-4d4f-9331-579871fd7983", "date": "Tue, 17 Dec 2024 21:31:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
1048	2024-12-17T21:31:06.483Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
1049	2024-12-17T21:31:06.485Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "da79c2e8-793d-4b2c-a875-61ca3f9876dc", "date": "Tue, 17 Dec 2024 21:31:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
1050	2024-12-17T21:31:06.485Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
1051	2024-12-17T21:31:06.487Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
1052	2024-12-17T21:31:06.487Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 551.073594ms iteration = 11 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
1053	2024-12-17T21:31:06.518Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1054	2024-12-17T21:31:06.519Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1055	2024-12-17T21:31:06.521Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 1a3e1a79-c78b-4fb4-921b-c5e1e32e8b40, rack_id: 8216c8d1-8011-434d-a57a-2b1ce2e89ad8, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(d6c2617a-6b89-4779-bc70-3002fb0a7e5e) } rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0
1056	2024-12-17T21:31:06.521Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1057	2024-12-17T21:31:06.521Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1058	2024-12-17T21:31:06.530Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
1059	2024-12-17T21:31:06.530Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ba2ae6e-5402-4eee-bfe5-d5acb153aefa", "content-length": "134", "date": "Tue, 17 Dec 2024 21:31:05 GMT"} })
1060	2024-12-17T21:31:06.530Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
1061	2024-12-17T21:31:06.530Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = d6c2617a-6b89-4779-bc70-3002fb0a7e5e
1062	2024-12-17T21:31:06.530Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
1063	2024-12-17T21:31:06.530Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
1064	2024-12-17T21:31:06.530Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
1065	2024-12-17T21:31:06.530Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
1066	2024-12-17T21:31:06.530Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
1067	2024-12-17T21:31:06.531Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
1068	2024-12-17T21:31:06.531Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
1069	2024-12-17T21:31:06.531Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
1070	2024-12-17T21:31:06.531Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
1071	2024-12-17T21:31:06.531Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1072	2024-12-17T21:31:06.531Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1073	2024-12-17T21:31:06.533Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
1074	2024-12-17T21:31:06.533Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "32111c9f-3ec4-4ed2-9d4b-35e8e2698bcb", "content-length": "2", "date": "Tue, 17 Dec 2024 21:31:05 GMT"} })
1075	2024-12-17T21:31:06.533Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
1076	2024-12-17T21:31:06.533Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "bfaaf557-c350-4ad5-a4bc-f96eb6662988", "content-length": "51", "date": "Tue, 17 Dec 2024 21:31:05 GMT"} })
1077	2024-12-17T21:31:06.533Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0
1078	2024-12-17T21:31:06.533Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
1079	2024-12-17T21:31:06.620Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "0e4f1760-e6e6-4f4e-bbcd-b01360ae9c0e", "date": "Tue, 17 Dec 2024 21:31:06 GMT"} })
1080	2024-12-17T21:31:06.620Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch = Switch0
1081	2024-12-17T21:31:06.620Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
1082	2024-12-17T21:31:06.620Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "18e74a0f-e18a-420e-a57c-3acf46f068a8", "content-length": "682", "date": "Tue, 17 Dec 2024 21:31:06 GMT"} })
1083	2024-12-17T21:31:06.620Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1084	2024-12-17T21:31:06.620Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1085	2024-12-17T21:31:06.624Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1086	2024-12-17T21:31:06.624Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1087	2024-12-17T21:31:06.627Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1088	2024-12-17T21:31:06.628Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1089	2024-12-17T21:31:06.630Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1090	2024-12-17T21:31:06.630Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1091	2024-12-17T21:31:06.638Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 701.979398ms iteration = 17 status = {}
1092	2024-12-17T21:31:10.407Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 277 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = 8ecb652a-703c-4032-93d3-664d608e1a95 response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
1093	2024-12-17T21:31:15.936Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 50 reason = Timeout
1094	2024-12-17T21:31:15.936Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1095	2024-12-17T21:31:15.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1096	2024-12-17T21:31:15.937Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 50 reason = Timeout
1097	2024-12-17T21:31:15.937Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1098	2024-12-17T21:31:15.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1099	2024-12-17T21:31:15.938Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1100	2024-12-17T21:31:15.939Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1101	2024-12-17T21:31:15.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1102	2024-12-17T21:31:15.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1103	2024-12-17T21:31:15.941Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1104	2024-12-17T21:31:15.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1105	2024-12-17T21:31:15.943Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1106	2024-12-17T21:31:15.943Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1107	2024-12-17T21:31:15.944Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1108	2024-12-17T21:31:15.947Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
1109	2024-12-17T21:31:15.947Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1110	2024-12-17T21:31:15.947Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1111	2024-12-17T21:31:15.958Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
1112	2024-12-17T21:31:15.958Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.077237ms iteration = 50 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
1113	2024-12-17T21:31:15.968Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.670567ms iteration = 50 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
1114	2024-12-17T21:31:20.408Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 304 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = fc61bf69-041c-4787-8184-91b62c8df579 response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
1115	2024-12-17T21:31:25.936Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 51 reason = Timeout
1116	2024-12-17T21:31:25.936Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 51 reason = Timeout
1117	2024-12-17T21:31:25.936Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1118	2024-12-17T21:31:25.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1119	2024-12-17T21:31:25.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1120	2024-12-17T21:31:25.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1121	2024-12-17T21:31:25.938Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1122	2024-12-17T21:31:25.939Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1123	2024-12-17T21:31:25.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1124	2024-12-17T21:31:25.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1125	2024-12-17T21:31:25.941Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1126	2024-12-17T21:31:25.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1127	2024-12-17T21:31:25.943Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1128	2024-12-17T21:31:25.943Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1129	2024-12-17T21:31:25.944Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1130	2024-12-17T21:31:25.946Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
1131	2024-12-17T21:31:25.946Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1132	2024-12-17T21:31:25.947Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1133	2024-12-17T21:31:25.955Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
1134	2024-12-17T21:31:25.955Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.411298ms iteration = 51 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
1135	2024-12-17T21:31:25.967Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 30.856825ms iteration = 51 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
1136	2024-12-17T21:31:30.407Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 335 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = 12e45d53-aee7-48d7-b89c-cc2f496dd3c4 response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
1137	2024-12-17T21:31:35.935Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 18 reason = Timeout
1138	2024-12-17T21:31:35.935Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
1139	2024-12-17T21:31:35.935Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 19 reason = Timeout
1140	2024-12-17T21:31:35.935Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 18 reason = Timeout
1141	2024-12-17T21:31:35.935Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1142	2024-12-17T21:31:35.935Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1143	2024-12-17T21:31:35.935Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 18 reason = Timeout
1144	2024-12-17T21:31:35.935Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1145	2024-12-17T21:31:35.935Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 18 reason = Timeout
1146	2024-12-17T21:31:35.935Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 18 reason = Timeout
1147	2024-12-17T21:31:35.935Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1148	2024-12-17T21:31:35.935Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 18 reason = Timeout
1149	2024-12-17T21:31:35.935Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 52 reason = Timeout
1150	2024-12-17T21:31:35.935Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
1151	2024-12-17T21:31:35.935Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 18 reason = Timeout
1152	2024-12-17T21:31:35.935Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
1153	2024-12-17T21:31:35.935Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1154	2024-12-17T21:31:35.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1155	2024-12-17T21:31:35.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1156	2024-12-17T21:31:35.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1157	2024-12-17T21:31:35.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1158	2024-12-17T21:31:35.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
1159	2024-12-17T21:31:35.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
1160	2024-12-17T21:31:35.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1161	2024-12-17T21:31:35.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1162	2024-12-17T21:31:35.936Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 18 reason = Timeout
1163	2024-12-17T21:31:35.936Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 18 reason = Timeout
1164	2024-12-17T21:31:35.936Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 18 reason = Timeout
1165	2024-12-17T21:31:35.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
1166	2024-12-17T21:31:35.936Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
1167	2024-12-17T21:31:35.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
1168	2024-12-17T21:31:35.936Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 18 reason = Timeout
1169	2024-12-17T21:31:35.936Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
1170	2024-12-17T21:31:35.936Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
1171	2024-12-17T21:31:35.936Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1172	2024-12-17T21:31:35.936Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 52 reason = Timeout
1173	2024-12-17T21:31:35.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1174	2024-12-17T21:31:35.936Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 18 reason = Timeout
1175	2024-12-17T21:31:35.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
1176	2024-12-17T21:31:35.936Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 18 reason = Timeout
1177	2024-12-17T21:31:35.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
1178	2024-12-17T21:31:35.936Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 18 reason = Timeout
1179	2024-12-17T21:31:35.936Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
1180	2024-12-17T21:31:35.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1181	2024-12-17T21:31:35.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
1182	2024-12-17T21:31:35.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
1183	2024-12-17T21:31:35.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
1184	2024-12-17T21:31:35.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1185	2024-12-17T21:31:35.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
1186	2024-12-17T21:31:35.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1187	2024-12-17T21:31:35.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
1188	2024-12-17T21:31:35.938Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1189	2024-12-17T21:31:35.938Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1754, tv_nsec: 70176785 } })
1190	2024-12-17T21:31:35.939Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1191	2024-12-17T21:31:35.939Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
1192	2024-12-17T21:31:35.939Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1193	2024-12-17T21:31:35.939Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.085891ms iteration = 18 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
1194	2024-12-17T21:31:35.940Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
1195	2024-12-17T21:31:35.940Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1196	2024-12-17T21:31:35.940Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1197	2024-12-17T21:31:35.940Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1754, tv_nsec: 72323883 } })
1198	2024-12-17T21:31:35.941Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
1199	2024-12-17T21:31:35.942Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
1200	2024-12-17T21:31:35.942Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1201	2024-12-17T21:31:35.942Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1202	2024-12-17T21:31:35.942Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
1203	2024-12-17T21:31:35.942Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1204	2024-12-17T21:31:35.942Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1205	2024-12-17T21:31:35.944Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1206	2024-12-17T21:31:35.945Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 8.330036ms iteration = 18 status = {"errors":[],"garbage_collect_requested":[]}
1207	2024-12-17T21:31:35.945Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1208	2024-12-17T21:31:35.946Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
1209	2024-12-17T21:31:35.946Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
1210	2024-12-17T21:31:35.946Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
1211	2024-12-17T21:31:35.946Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
1212	2024-12-17T21:31:35.946Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1213	2024-12-17T21:31:35.946Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
1214	2024-12-17T21:31:35.946Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1215	2024-12-17T21:31:35.947Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 11.001947ms iteration = 18 status = {"errors":[],"records_set_to_done":[]}
1216	2024-12-17T21:31:35.948Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
1217	2024-12-17T21:31:35.948Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
1218	2024-12-17T21:31:35.948Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1219	2024-12-17T21:31:35.948Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
1220	2024-12-17T21:31:35.948Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1221	2024-12-17T21:31:35.951Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1222	2024-12-17T21:31:35.951Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1223	2024-12-17T21:31:35.951Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1224	2024-12-17T21:31:35.951Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1225	2024-12-17T21:31:35.952Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1226	2024-12-17T21:31:35.952Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1754, tv_nsec: 83876626 } })
1227	2024-12-17T21:31:35.952Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
1228	2024-12-17T21:31:35.952Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
1229	2024-12-17T21:31:35.952Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
1230	2024-12-17T21:31:35.952Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
1231	2024-12-17T21:31:35.952Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
1232	2024-12-17T21:31:35.953Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1233	2024-12-17T21:31:35.954Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
1234	2024-12-17T21:31:35.954Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1235	2024-12-17T21:31:35.954Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1236	2024-12-17T21:31:35.955Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
1237	2024-12-17T21:31:35.955Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1238	2024-12-17T21:31:35.955Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1239	2024-12-17T21:31:35.982Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
1240	2024-12-17T21:31:35.982Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
1241	2024-12-17T21:31:35.985Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 49.677155ms iteration = 18 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
1242	2024-12-17T21:31:35.986Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1243	2024-12-17T21:31:35.987Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1244	2024-12-17T21:31:35.989Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1245	2024-12-17T21:31:35.989Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1246	2024-12-17T21:31:35.989Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1247	2024-12-17T21:31:35.990Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 53.348734ms iteration = 52 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
1248	2024-12-17T21:31:35.991Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1249	2024-12-17T21:31:35.992Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
1250	2024-12-17T21:31:35.992Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
1251	2024-12-17T21:31:35.993Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1252	2024-12-17T21:31:35.993Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1253	2024-12-17T21:31:35.993Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1254	2024-12-17T21:31:35.994Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: d3278cd4-2587-4f0f-89dd-b4d4669a6ffe, lookup_type: ById(d3278cd4-2587-4f0f-89dd-b4d4669a6ffe) } result = Ok(())
1255	2024-12-17T21:31:35.994Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1256	2024-12-17T21:31:35.994Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 57.27628ms iteration = 18 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
1257	2024-12-17T21:31:35.994Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1258	2024-12-17T21:31:35.994Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
1259	2024-12-17T21:31:35.994Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 57.540637ms iteration = 18 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
1260	2024-12-17T21:31:35.994Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1261	2024-12-17T21:31:35.994Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1262	2024-12-17T21:31:35.996Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 60.355034ms iteration = 18 status = 0
1263	2024-12-17T21:31:35.996Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1264	2024-12-17T21:31:35.998Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
1265	2024-12-17T21:31:35.998Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
1266	2024-12-17T21:31:35.998Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
1267	2024-12-17T21:31:35.998Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 61.614107ms iteration = 18 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
1268	2024-12-17T21:31:35.998Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1269	2024-12-17T21:31:35.999Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
1270	2024-12-17T21:31:35.999Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1271	2024-12-17T21:31:35.999Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1272	2024-12-17T21:31:36.000Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
1273	2024-12-17T21:31:36.000Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1274	2024-12-17T21:31:36.012Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
1275	2024-12-17T21:31:36.012Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 76.672935ms iteration = 52 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
1276	2024-12-17T21:31:36.030Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 93.693956ms iteration = 18 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
1277	2024-12-17T21:31:36.043Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1278	2024-12-17T21:31:36.043Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1279	2024-12-17T21:31:36.054Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 zone_address = fd00:1122:3344:101::2
1280	2024-12-17T21:31:36.054Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1281	2024-12-17T21:31:36.054Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "aee353da-715a-4f06-80c8-ffe6add3f718", "content-length": "26", "date": "Tue, 17 Dec 2024 21:31:35 GMT"} })
1282	2024-12-17T21:31:36.054Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1283	2024-12-17T21:31:36.056Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
1284	2024-12-17T21:31:36.061Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 125.926198ms iteration = 19 status = {"physical_disks_added":0}
1285	2024-12-17T21:31:36.131Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1286	2024-12-17T21:31:36.132Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1287	2024-12-17T21:31:36.133Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1288	2024-12-17T21:31:36.135Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
1289	2024-12-17T21:31:36.135Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1290	2024-12-17T21:31:36.135Z	INFO	nexus (BackgroundTasks): VPC route manager sled d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
1291	2024-12-17T21:31:36.135Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
1292	2024-12-17T21:31:36.135Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1293	2024-12-17T21:31:36.135Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "b90b8176-6891-45e9-8c15-9dc88c3dddea", "content-length": "372", "date": "Tue, 17 Dec 2024 21:31:36 GMT"} })
1294	2024-12-17T21:31:36.135Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1295	2024-12-17T21:31:36.136Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1296	2024-12-17T21:31:36.136Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1297	2024-12-17T21:31:36.139Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
1298	2024-12-17T21:31:36.139Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1299	2024-12-17T21:31:36.139Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1300	2024-12-17T21:31:36.140Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
1301	2024-12-17T21:31:36.140Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1302	2024-12-17T21:31:36.141Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ddace650-6295-40b7-832b-9fb9f8afa5d7", "content-length": "26", "date": "Tue, 17 Dec 2024 21:31:35 GMT"} })
1303	2024-12-17T21:31:36.141Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1304	2024-12-17T21:31:36.142Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
1305	2024-12-17T21:31:36.142Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1306	2024-12-17T21:31:36.142Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "546f8f22-0374-40b5-ba9a-9d3498da50c8", "content-length": "26", "date": "Tue, 17 Dec 2024 21:31:35 GMT"} })
1307	2024-12-17T21:31:36.142Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1308	2024-12-17T21:31:36.143Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
1309	2024-12-17T21:31:36.144Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
1310	2024-12-17T21:31:36.150Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
1311	2024-12-17T21:31:36.150Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
1312	2024-12-17T21:31:36.153Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "c025f5d4-218b-483b-9ea6-dfa66bffd413", "date": "Tue, 17 Dec 2024 21:31:36 GMT"} })
1313	2024-12-17T21:31:36.153Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1314	2024-12-17T21:31:36.153Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1315	2024-12-17T21:31:36.156Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1316	2024-12-17T21:31:36.156Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1317	2024-12-17T21:31:36.158Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1318	2024-12-17T21:31:36.158Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1319	2024-12-17T21:31:36.241Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
1320	2024-12-17T21:31:36.241Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
1321	2024-12-17T21:31:36.241Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "d956c56e-6f9a-45bb-936e-5c5d567d8512", "content-length": "691", "date": "Tue, 17 Dec 2024 21:31:35 GMT"} })
1322	2024-12-17T21:31:36.241Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
1323	2024-12-17T21:31:36.241Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
1324	2024-12-17T21:31:36.241Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
1325	2024-12-17T21:31:36.241Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "bf82351d-e6f3-431b-99ce-b562d0381e8a", "content-length": "1", "date": "Tue, 17 Dec 2024 21:31:35 GMT"} })
1326	2024-12-17T21:31:36.245Z	INFO	nexus (BackgroundTasks): VPC route manager sled d3278cd4-2587-4f0f-89dd-b4d4669a6ffe push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
1327	2024-12-17T21:31:36.245Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
1328	2024-12-17T21:31:36.246Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "992ef39a-a878-4e9f-897e-a60021fb3786", "date": "Tue, 17 Dec 2024 21:31:35 GMT"} })
1329	2024-12-17T21:31:36.246Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
1330	2024-12-17T21:31:36.246Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1331	2024-12-17T21:31:36.247Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 312.122987ms iteration = 18 status = {}
1332	2024-12-17T21:31:36.248Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 312.471256ms iteration = 18 status = null
1333	2024-12-17T21:31:36.248Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 312.749685ms iteration = 18 status = {}
1334	2024-12-17T21:31:36.249Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 312.392434ms iteration = 18 status = {}
1335	2024-12-17T21:31:36.281Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1336	2024-12-17T21:31:36.281Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1337	2024-12-17T21:31:36.283Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 1a3e1a79-c78b-4fb4-921b-c5e1e32e8b40, rack_id: 8216c8d1-8011-434d-a57a-2b1ce2e89ad8, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(d6c2617a-6b89-4779-bc70-3002fb0a7e5e) } rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0
1338	2024-12-17T21:31:36.283Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1339	2024-12-17T21:31:36.283Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1340	2024-12-17T21:31:36.292Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
1341	2024-12-17T21:31:36.292Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "f7c204f0-6bcd-4832-ac7d-2af8f2efca9e", "content-length": "134", "date": "Tue, 17 Dec 2024 21:31:35 GMT"} })
1342	2024-12-17T21:31:36.292Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
1343	2024-12-17T21:31:36.292Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = d6c2617a-6b89-4779-bc70-3002fb0a7e5e
1344	2024-12-17T21:31:36.293Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
1345	2024-12-17T21:31:36.293Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
1346	2024-12-17T21:31:36.293Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
1347	2024-12-17T21:31:36.293Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
1348	2024-12-17T21:31:36.293Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
1349	2024-12-17T21:31:36.293Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
1350	2024-12-17T21:31:36.293Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
1351	2024-12-17T21:31:36.293Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
1352	2024-12-17T21:31:36.293Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
1353	2024-12-17T21:31:36.293Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1354	2024-12-17T21:31:36.293Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1355	2024-12-17T21:31:36.296Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
1356	2024-12-17T21:31:36.296Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "e233e77b-be3c-4be2-a28f-5d4ef055c3fb", "content-length": "2", "date": "Tue, 17 Dec 2024 21:31:35 GMT"} })
1357	2024-12-17T21:31:36.296Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
1358	2024-12-17T21:31:36.296Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "47169041-1e95-4bb4-afba-34b7230293d5", "content-length": "51", "date": "Tue, 17 Dec 2024 21:31:35 GMT"} })
1359	2024-12-17T21:31:36.296Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0
1360	2024-12-17T21:31:36.296Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
1361	2024-12-17T21:31:36.394Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "5cec8822-3076-4c0f-aecd-e9c7a7261f7a", "date": "Tue, 17 Dec 2024 21:31:35 GMT"} })
1362	2024-12-17T21:31:36.394Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch = Switch0
1363	2024-12-17T21:31:36.394Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
1364	2024-12-17T21:31:36.395Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a2cd16b-2a45-41b3-aeca-30f5885040d1", "content-length": "682", "date": "Tue, 17 Dec 2024 21:31:35 GMT"} })
1365	2024-12-17T21:31:36.395Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1366	2024-12-17T21:31:36.395Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1367	2024-12-17T21:31:36.399Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1368	2024-12-17T21:31:36.399Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1369	2024-12-17T21:31:36.402Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1370	2024-12-17T21:31:36.403Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1371	2024-12-17T21:31:36.404Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1372	2024-12-17T21:31:36.405Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1373	2024-12-17T21:31:36.412Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 476.611479ms iteration = 18 status = {}
1374	2024-12-17T21:31:36.561Z	DEBG	nexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
1375	2024-12-17T21:31:36.561Z	DEBG	nexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
1376	2024-12-17T21:31:36.591Z	DEBG	nexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
1377	2024-12-17T21:31:36.591Z	INFO	nexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:36048
1378	2024-12-17T21:31:36.591Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:36048 req_id = a4a41368-5fc1-482e-8706-8425077760f4 roles = RoleSet { roles: {} } uri = /metrics/producers
1379	2024-12-17T21:31:36.592Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:36048 req_id = a4a41368-5fc1-482e-8706-8425077760f4 resource = Database result = Ok(()) uri = /metrics/producers
1380	2024-12-17T21:31:36.632Z	INFO	nexus: assigned collector to new producer collector_id = 952238fd-7df7-4f2b-b8ee-8be2d7488464 file = nexus/src/app/oximeter.rs:76 producer_id = 2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
1381	2024-12-17T21:31:36.632Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 40430 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:36048 req_id = a4a41368-5fc1-482e-8706-8425077760f4 response_code = 201 uri = /metrics/producers
1382	2024-12-17T21:31:36.632Z	DEBG	nexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "a4a41368-5fc1-482e-8706-8425077760f4", "content-length": "41", "date": "Tue, 17 Dec 2024 21:31:36 GMT"} })
1383	2024-12-17T21:31:36.633Z	DEBG	nexus (producer-registration-task): registered with nexus successfully
1384	2024-12-17T21:31:36.633Z	DEBG	nexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
1385	2024-12-17T21:31:40.395Z	INFO	nexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:50255
1386	2024-12-17T21:31:40.395Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = d8370f47-49b1-4d44-8aab-e476641d94c5 roles = RoleSet { roles: {} } uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?sort_by=id_ascending
1387	2024-12-17T21:31:40.396Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = d8370f47-49b1-4d44-8aab-e476641d94c5 resource = Database result = Ok(()) uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?sort_by=id_ascending
1388	2024-12-17T21:31:40.398Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3029 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = d8370f47-49b1-4d44-8aab-e476641d94c5 response_code = 200 uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?sort_by=id_ascending
1389	2024-12-17T21:31:40.398Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 9c8244fd-3698-4e12-8d2a-468293cf3025 roles = RoleSet { roles: {} } uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMTkzZjYyMi00MGY1LTQyOGYtYmVhMy01MmNjNjdlNzhhMzkifX0%3D
1390	2024-12-17T21:31:40.399Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 9c8244fd-3698-4e12-8d2a-468293cf3025 resource = Database result = Ok(()) uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMTkzZjYyMi00MGY1LTQyOGYtYmVhMy01MmNjNjdlNzhhMzkifX0%3D
1391	2024-12-17T21:31:40.400Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2071 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 9c8244fd-3698-4e12-8d2a-468293cf3025 response_code = 200 uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMTkzZjYyMi00MGY1LTQyOGYtYmVhMy01MmNjNjdlNzhhMzkifX0%3D
1392	2024-12-17T21:31:40.407Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 270 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = 7bcb0d77-54f2-4b90-949d-8fdf4b28f090 response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
1393	2024-12-17T21:31:45.935Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 53 reason = Timeout
1394	2024-12-17T21:31:45.935Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1395	2024-12-17T21:31:45.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1396	2024-12-17T21:31:45.936Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 53 reason = Timeout
1397	2024-12-17T21:31:45.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1398	2024-12-17T21:31:45.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1399	2024-12-17T21:31:45.938Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1400	2024-12-17T21:31:45.940Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1401	2024-12-17T21:31:45.940Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1402	2024-12-17T21:31:45.940Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1403	2024-12-17T21:31:45.942Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1404	2024-12-17T21:31:45.942Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1405	2024-12-17T21:31:45.944Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1406	2024-12-17T21:31:45.946Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
1407	2024-12-17T21:31:45.946Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1408	2024-12-17T21:31:45.946Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1409	2024-12-17T21:31:45.955Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
1410	2024-12-17T21:31:45.955Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.152715ms iteration = 53 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
1411	2024-12-17T21:31:45.969Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1412	2024-12-17T21:31:45.969Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1413	2024-12-17T21:31:45.971Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 34.293268ms iteration = 53 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
1414	2024-12-17T21:31:50.407Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 255 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = 9ad88d30-b1ce-466a-803e-37b7be674118 response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
1415	2024-12-17T21:31:55.935Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 54 reason = Timeout
1416	2024-12-17T21:31:55.935Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1417	2024-12-17T21:31:55.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1418	2024-12-17T21:31:55.937Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 54 reason = Timeout
1419	2024-12-17T21:31:55.937Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1420	2024-12-17T21:31:55.937Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1421	2024-12-17T21:31:55.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1422	2024-12-17T21:31:55.938Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1423	2024-12-17T21:31:55.938Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1424	2024-12-17T21:31:55.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1425	2024-12-17T21:31:55.939Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1426	2024-12-17T21:31:55.940Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1427	2024-12-17T21:31:55.942Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1428	2024-12-17T21:31:55.942Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1429	2024-12-17T21:31:55.943Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1430	2024-12-17T21:31:55.945Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
1431	2024-12-17T21:31:55.945Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1432	2024-12-17T21:31:55.946Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1433	2024-12-17T21:31:55.957Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
1434	2024-12-17T21:31:55.957Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.065297ms iteration = 54 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
1435	2024-12-17T21:31:55.967Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 29.881069ms iteration = 54 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
1436	2024-12-17T21:32:00.407Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 308 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = 2ee52041-e190-4872-8d60-e6d94d410214 response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
1437	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 5 reason = Timeout
1438	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 10 reason = Timeout
1439	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1440	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
1441	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1442	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = external_endpoints iteration = 10 reason = Timeout
1443	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 19 reason = Timeout
1444	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
1445	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1446	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 10 reason = Timeout
1447	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 12 reason = Timeout
1448	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 10 reason = Timeout
1449	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
1450	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1451	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 10 reason = Timeout
1452	2024-12-17T21:32:05.936Z	INFO	nexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-17 21:22:05.936143579 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
1453	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 55 reason = Timeout
1454	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
1455	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 10 reason = Timeout
1456	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1457	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 19 reason = Timeout
1458	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 19 reason = Timeout
1459	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1460	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 12 reason = Timeout
1461	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 20 reason = Timeout
1462	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 19 reason = Timeout
1463	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1464	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1465	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 10 reason = Timeout
1466	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 11 reason = Timeout
1467	2024-12-17T21:32:05.936Z	INFO	nexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
1468	2024-12-17T21:32:05.936Z	WARN	nexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5
1469	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
1470	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 13.65\u{b5}s iteration = 11 status = {"enabled":false,"target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5"}
1471	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 19 reason = Timeout
1472	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
1473	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 19 reason = Timeout
1474	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 19 reason = Timeout
1475	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
1476	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1477	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 19 reason = Timeout
1478	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
1479	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 10 reason = Timeout
1480	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
1481	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 10 reason = Timeout
1482	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
1483	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 19 reason = Timeout
1484	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
1485	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 19 reason = Timeout
1486	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 19 reason = Timeout
1487	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
1488	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
1489	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
1490	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 19 reason = Timeout
1491	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
1492	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 19 reason = Timeout
1493	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
1494	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 19 reason = Timeout
1495	2024-12-17T21:32:05.936Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
1496	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 10 reason = Timeout
1497	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 55 reason = Timeout
1498	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
1499	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1500	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1501	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1502	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1503	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1504	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
1505	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1506	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1507	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1508	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1509	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
1510	2024-12-17T21:32:05.936Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0005c310-624c-4312-b0fd-35f571746851.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6ac4d5e8-fcd9-4aa9-aed5-fb3dc293aac2.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9b804364-993d-47ba-81a1-d0ef32f96529.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("9b804364-993d-47ba-81a1-d0ef32f96529.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1784, tv_nsec: 68532990 } })
1511	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1512	2024-12-17T21:32:05.936Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("5dff2793-3ce8-4251-ba71-1a446966cbf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a761a47e-08b9-41c4-9bd8-8d77651dfa62.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a761a47e-08b9-41c4-9bd8-8d77651dfa62.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1784, tv_nsec: 68554211 } })
1513	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
1514	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
1515	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1516	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1517	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
1518	2024-12-17T21:32:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1519	2024-12-17T21:32:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
1520	2024-12-17T21:32:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
1521	2024-12-17T21:32:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1522	2024-12-17T21:32:05.937Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1523	2024-12-17T21:32:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1524	2024-12-17T21:32:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
1525	2024-12-17T21:32:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
1526	2024-12-17T21:32:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
1527	2024-12-17T21:32:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1528	2024-12-17T21:32:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
1529	2024-12-17T21:32:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
1530	2024-12-17T21:32:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1531	2024-12-17T21:32:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1532	2024-12-17T21:32:05.937Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1533	2024-12-17T21:32:05.937Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
1534	2024-12-17T21:32:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1535	2024-12-17T21:32:05.937Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1536	2024-12-17T21:32:05.937Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.382838ms iteration = 10 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
1537	2024-12-17T21:32:05.937Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
1538	2024-12-17T21:32:05.937Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.368457ms iteration = 10 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
1539	2024-12-17T21:32:05.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1540	2024-12-17T21:32:05.938Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1541	2024-12-17T21:32:05.938Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1542	2024-12-17T21:32:05.938Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1543	2024-12-17T21:32:05.940Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.835804ms iteration = 19 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
1544	2024-12-17T21:32:05.940Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1545	2024-12-17T21:32:05.940Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1546	2024-12-17T21:32:05.940Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1784, tv_nsec: 72577620 } })
1547	2024-12-17T21:32:05.940Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
1548	2024-12-17T21:32:05.940Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1549	2024-12-17T21:32:05.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1550	2024-12-17T21:32:05.941Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
1551	2024-12-17T21:32:05.941Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1552	2024-12-17T21:32:05.942Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
1553	2024-12-17T21:32:05.942Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1554	2024-12-17T21:32:05.942Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1555	2024-12-17T21:32:05.943Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1556	2024-12-17T21:32:05.944Z	INFO	nexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-17 21:22:05.936143579 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
1557	2024-12-17T21:32:05.944Z	DEBG	nexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 8.014497ms iteration = 10 status = {"expiration":"2024-12-17T21:22:05.936143579Z","pruned":[]}
1558	2024-12-17T21:32:05.944Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1559	2024-12-17T21:32:05.945Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1784, tv_nsec: 76725552 } })
1560	2024-12-17T21:32:05.945Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
1561	2024-12-17T21:32:05.946Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
1562	2024-12-17T21:32:05.946Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1563	2024-12-17T21:32:05.946Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
1564	2024-12-17T21:32:05.946Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1565	2024-12-17T21:32:05.946Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
1566	2024-12-17T21:32:05.947Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
1567	2024-12-17T21:32:05.947Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
1568	2024-12-17T21:32:05.947Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
1569	2024-12-17T21:32:05.948Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1570	2024-12-17T21:32:05.949Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 12.788306ms iteration = 19 status = {"errors":[],"records_set_to_done":[]}
1571	2024-12-17T21:32:05.949Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1572	2024-12-17T21:32:05.950Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1573	2024-12-17T21:32:05.951Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
1574	2024-12-17T21:32:05.951Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1575	2024-12-17T21:32:05.951Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1576	2024-12-17T21:32:05.952Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
1577	2024-12-17T21:32:05.952Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1578	2024-12-17T21:32:05.952Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1579	2024-12-17T21:32:05.952Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1580	2024-12-17T21:32:05.952Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1581	2024-12-17T21:32:05.953Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
1582	2024-12-17T21:32:05.953Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1583	2024-12-17T21:32:05.954Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1584	2024-12-17T21:32:05.957Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1585	2024-12-17T21:32:05.957Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1586	2024-12-17T21:32:05.958Z	DEBG	nexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 21.702307ms iteration = 10 status = {"errors":[],"found_port_ok":[]}
1587	2024-12-17T21:32:05.959Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1588	2024-12-17T21:32:05.959Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1589	2024-12-17T21:32:05.961Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
1590	2024-12-17T21:32:05.961Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1591	2024-12-17T21:32:05.961Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1592	2024-12-17T21:32:05.962Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
1593	2024-12-17T21:32:05.962Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1594	2024-12-17T21:32:05.962Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1595	2024-12-17T21:32:05.963Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1596	2024-12-17T21:32:05.965Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
1597	2024-12-17T21:32:05.965Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1598	2024-12-17T21:32:05.965Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1599	2024-12-17T21:32:05.966Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1600	2024-12-17T21:32:05.966Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1601	2024-12-17T21:32:05.969Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1602	2024-12-17T21:32:05.969Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1784, tv_nsec: 101611466 } })
1603	2024-12-17T21:32:05.969Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1604	2024-12-17T21:32:05.970Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
1605	2024-12-17T21:32:05.971Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 35.196462ms iteration = 55 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
1606	2024-12-17T21:32:05.972Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1607	2024-12-17T21:32:05.972Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1608	2024-12-17T21:32:05.976Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1609	2024-12-17T21:32:05.978Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
1610	2024-12-17T21:32:05.978Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1611	2024-12-17T21:32:05.979Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1612	2024-12-17T21:32:05.990Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1613	2024-12-17T21:32:05.993Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1614	2024-12-17T21:32:05.994Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1615	2024-12-17T21:32:05.994Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1616	2024-12-17T21:32:06.001Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 65.448272ms iteration = 20 status = {"physical_disks_added":0}
1617	2024-12-17T21:32:06.004Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 67.95913ms iteration = 19 status = {"errors":[],"garbage_collect_requested":[]}
1618	2024-12-17T21:32:06.004Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1619	2024-12-17T21:32:06.004Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1620	2024-12-17T21:32:06.007Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1621	2024-12-17T21:32:06.007Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1622	2024-12-17T21:32:06.010Z	DEBG	nexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
1623	2024-12-17T21:32:06.010Z	INFO	nexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
1624	2024-12-17T21:32:06.010Z	DEBG	nexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 73.949731ms iteration = 10 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
1625	2024-12-17T21:32:06.010Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
1626	2024-12-17T21:32:06.010Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1627	2024-12-17T21:32:06.011Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
1628	2024-12-17T21:32:06.011Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1629	2024-12-17T21:32:06.011Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
1630	2024-12-17T21:32:06.011Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1631	2024-12-17T21:32:06.011Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
1632	2024-12-17T21:32:06.011Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
1633	2024-12-17T21:32:06.011Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
1634	2024-12-17T21:32:06.011Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
1635	2024-12-17T21:32:06.013Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
1636	2024-12-17T21:32:06.013Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1637	2024-12-17T21:32:06.014Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1638	2024-12-17T21:32:06.016Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 79.849351ms iteration = 19 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
1639	2024-12-17T21:32:06.017Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
1640	2024-12-17T21:32:06.017Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1641	2024-12-17T21:32:06.017Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1642	2024-12-17T21:32:06.018Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1643	2024-12-17T21:32:06.018Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1644	2024-12-17T21:32:06.018Z	DEBG	nexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 82.719269ms iteration = 5 status = {"nsuccess":5}
1645	2024-12-17T21:32:06.019Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1646	2024-12-17T21:32:06.022Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
1647	2024-12-17T21:32:06.022Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1648	2024-12-17T21:32:06.022Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1649	2024-12-17T21:32:06.022Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1650	2024-12-17T21:32:06.023Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1651	2024-12-17T21:32:06.023Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
1652	2024-12-17T21:32:06.023Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
1653	2024-12-17T21:32:06.023Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
1654	2024-12-17T21:32:06.024Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
1655	2024-12-17T21:32:06.024Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1656	2024-12-17T21:32:06.024Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1657	2024-12-17T21:32:06.024Z	DEBG	nexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
1658	2024-12-17T21:32:06.024Z	INFO	nexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
1659	2024-12-17T21:32:06.024Z	DEBG	nexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 88.404412ms iteration = 10 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
1660	2024-12-17T21:32:06.027Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1661	2024-12-17T21:32:06.027Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1662	2024-12-17T21:32:06.028Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1663	2024-12-17T21:32:06.032Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
1664	2024-12-17T21:32:06.032Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1665	2024-12-17T21:32:06.032Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1666	2024-12-17T21:32:06.034Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1667	2024-12-17T21:32:06.035Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1668	2024-12-17T21:32:06.038Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
1669	2024-12-17T21:32:06.038Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 102.449873ms iteration = 55 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
1670	2024-12-17T21:32:06.040Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1671	2024-12-17T21:32:06.041Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
1672	2024-12-17T21:32:06.041Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1673	2024-12-17T21:32:06.041Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1674	2024-12-17T21:32:06.046Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
1675	2024-12-17T21:32:06.046Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
1676	2024-12-17T21:32:06.058Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
1677	2024-12-17T21:32:06.060Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
1678	2024-12-17T21:32:06.064Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 128.105647ms iteration = 19 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
1679	2024-12-17T21:32:06.067Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 130.943013ms iteration = 19 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
1680	2024-12-17T21:32:06.067Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
1681	2024-12-17T21:32:06.067Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
1682	2024-12-17T21:32:06.067Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
1683	2024-12-17T21:32:06.067Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
1684	2024-12-17T21:32:06.068Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1685	2024-12-17T21:32:06.070Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
1686	2024-12-17T21:32:06.070Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 134.062698ms iteration = 19 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
1687	2024-12-17T21:32:06.071Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: d3278cd4-2587-4f0f-89dd-b4d4669a6ffe, lookup_type: ById(d3278cd4-2587-4f0f-89dd-b4d4669a6ffe) } result = Ok(())
1688	2024-12-17T21:32:06.071Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1689	2024-12-17T21:32:06.071Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1690	2024-12-17T21:32:06.073Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
1691	2024-12-17T21:32:06.073Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
1692	2024-12-17T21:32:06.073Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
1693	2024-12-17T21:32:06.073Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1694	2024-12-17T21:32:06.073Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1695	2024-12-17T21:32:06.073Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 137.468671ms iteration = 19 status = 0
1696	2024-12-17T21:32:06.074Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
1697	2024-12-17T21:32:06.074Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
1698	2024-12-17T21:32:06.074Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
1699	2024-12-17T21:32:06.074Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1700	2024-12-17T21:32:06.074Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1701	2024-12-17T21:32:06.075Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1702	2024-12-17T21:32:06.076Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1703	2024-12-17T21:32:06.078Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1704	2024-12-17T21:32:06.078Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1705	2024-12-17T21:32:06.079Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1706	2024-12-17T21:32:06.080Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1707	2024-12-17T21:32:06.080Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1708	2024-12-17T21:32:06.080Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1709	2024-12-17T21:32:06.082Z	DEBG	nexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-17T21:23:03.821125Z, time_modified: 2024-12-17T21:23:03.821125Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 81e78e1e-e55a-44f9-abdc-c9c916e1d9fd, db_silo: Silo { identity: SiloIdentity { id: 81e78e1e-e55a-44f9-abdc-c9c916e1d9fd, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-17T21:23:49.443930Z, time_modified: 2024-12-17T21:23:49.443930Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e275643b21ab02733b9616c9cd3b0193e27c7f6294b7d310d6f1726c72768e50" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 81e78e1e-e55a-44f9-abdc-c9c916e1d9fd, db_silo: Silo { identity: SiloIdentity { id: 81e78e1e-e55a-44f9-abdc-c9c916e1d9fd, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-17T21:23:49.443930Z, time_modified: 2024-12-17T21:23:49.443930Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e275643b21ab02733b9616c9cd3b0193e27c7f6294b7d310d6f1726c72768e50" }] }) }
1710	2024-12-17T21:32:06.082Z	DEBG	nexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 146.476324ms iteration = 10 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"81e78e1e-e55a-44f9-abdc-c9c916e1d9fd","tls_certs":["e275643b21ab02733b9616c9cd3b0193e27c7f6294b7d310d6f1726c72768e50"]}},"default_endpoint":{"silo_id":"81e78e1e-e55a-44f9-abdc-c9c916e1d9fd","tls_certs":["e275643b21ab02733b9616c9cd3b0193e27c7f6294b7d310d6f1726c72768e50"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
1711	2024-12-17T21:32:06.094Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
1712	2024-12-17T21:32:06.094Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1713	2024-12-17T21:32:06.094Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d31278d0-1101-4366-a6be-ee2f8ec36a5b", "content-length": "26", "date": "Tue, 17 Dec 2024 21:32:05 GMT"} })
1714	2024-12-17T21:32:06.094Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1715	2024-12-17T21:32:06.097Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
1716	2024-12-17T21:32:06.106Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
1717	2024-12-17T21:32:06.106Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1718	2024-12-17T21:32:06.106Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f2dc724-df7a-413f-be5a-5f50c866c13c", "content-length": "26", "date": "Tue, 17 Dec 2024 21:32:05 GMT"} })
1719	2024-12-17T21:32:06.106Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1720	2024-12-17T21:32:06.108Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
1721	2024-12-17T21:32:06.109Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
1722	2024-12-17T21:32:06.109Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
1723	2024-12-17T21:32:06.109Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 172.956171ms iteration = 19 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
1724	2024-12-17T21:32:06.113Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1725	2024-12-17T21:32:06.114Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1726	2024-12-17T21:32:06.119Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
1727	2024-12-17T21:32:06.119Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = ea6acdde-77d4-4bf6-b578-d9a0a29ac37b dns_zone_name = oxide.test found_so_far = 0 version = 2
1728	2024-12-17T21:32:06.119Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1729	2024-12-17T21:32:06.119Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1730	2024-12-17T21:32:06.121Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1731	2024-12-17T21:32:06.123Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
1732	2024-12-17T21:32:06.123Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 574f0c35-b8f2-48fe-831c-78afc4a62aaa dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
1733	2024-12-17T21:32:06.123Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1734	2024-12-17T21:32:06.123Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1735	2024-12-17T21:32:06.125Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1736	2024-12-17T21:32:06.125Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1737	2024-12-17T21:32:06.125Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1738	2024-12-17T21:32:06.125Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1739	2024-12-17T21:32:06.128Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1740	2024-12-17T21:32:06.128Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1741	2024-12-17T21:32:06.128Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = ea6acdde-77d4-4bf6-b578-d9a0a29ac37b dns_zone_name = oxide.test found_so_far = 1 version = 2
1742	2024-12-17T21:32:06.128Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
1743	2024-12-17T21:32:06.128Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-17 21:23:49.491999 UTC dns_group = external generation = 2
1744	2024-12-17T21:32:06.128Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 192.435888ms iteration = 10 status = {"generation":2}
1745	2024-12-17T21:32:06.128Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1746	2024-12-17T21:32:06.140Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 zone_address = fd00:1122:3344:101::2
1747	2024-12-17T21:32:06.140Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1748	2024-12-17T21:32:06.140Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f1f612af-2c1d-41a0-a99d-c335ec616fc0", "content-length": "26", "date": "Tue, 17 Dec 2024 21:32:05 GMT"} })
1749	2024-12-17T21:32:06.141Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1750	2024-12-17T21:32:06.143Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
1751	2024-12-17T21:32:06.144Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1752	2024-12-17T21:32:06.147Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
1753	2024-12-17T21:32:06.147Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1754	2024-12-17T21:32:06.148Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1755	2024-12-17T21:32:06.166Z	WARN	nexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
1756	2024-12-17T21:32:06.166Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 230.138938ms iteration = 10 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
1757	2024-12-17T21:32:06.202Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
1758	2024-12-17T21:32:06.213Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
1759	2024-12-17T21:32:06.214Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "ce1f30bc-dc2c-436f-aefb-04c30b731f41", "date": "Tue, 17 Dec 2024 21:32:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
1760	2024-12-17T21:32:06.214Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
1761	2024-12-17T21:32:06.216Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "ac7bda2d-f7b8-4f4a-b46d-7d273f35da32", "date": "Tue, 17 Dec 2024 21:32:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
1762	2024-12-17T21:32:06.216Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
1763	2024-12-17T21:32:06.218Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
1764	2024-12-17T21:32:06.218Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 282.125285ms iteration = 12 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
1765	2024-12-17T21:32:06.245Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 308.927491ms iteration = 19 status = {}
1766	2024-12-17T21:32:06.261Z	INFO	nexus (BackgroundTasks): VPC route manager sled d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
1767	2024-12-17T21:32:06.261Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
1768	2024-12-17T21:32:06.262Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "acb1650b-8846-46cd-a7ac-efa988aa86c3", "content-length": "372", "date": "Tue, 17 Dec 2024 21:32:06 GMT"} })
1769	2024-12-17T21:32:06.262Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1770	2024-12-17T21:32:06.262Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1771	2024-12-17T21:32:06.264Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1772	2024-12-17T21:32:06.264Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 574f0c35-b8f2-48fe-831c-78afc4a62aaa dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
1773	2024-12-17T21:32:06.264Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
1774	2024-12-17T21:32:06.264Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-17 21:23:48.446255 UTC dns_group = internal generation = 1
1775	2024-12-17T21:32:06.264Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 328.680065ms iteration = 10 status = {"generation":1}
1776	2024-12-17T21:32:06.266Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
1777	2024-12-17T21:32:06.266Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1778	2024-12-17T21:32:06.266Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1779	2024-12-17T21:32:06.267Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
1780	2024-12-17T21:32:06.268Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9ce3012-bb7e-45d8-abc5-cb0c7b1f4923", "content-length": "1", "date": "Tue, 17 Dec 2024 21:32:06 GMT"} })
1781	2024-12-17T21:32:06.269Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
1782	2024-12-17T21:32:06.270Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1783	2024-12-17T21:32:06.274Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 338.679775ms iteration = 19 status = null
1784	2024-12-17T21:32:06.311Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
1785	2024-12-17T21:32:06.311Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "64d7ace5-5be8-4d12-bb12-1ee63ee28ee0", "date": "Tue, 17 Dec 2024 21:32:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
1786	2024-12-17T21:32:06.311Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
1787	2024-12-17T21:32:06.312Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
1788	2024-12-17T21:32:06.312Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
1789	2024-12-17T21:32:06.313Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "25afb07d-9f67-4c8e-acaf-6e8e8a59b64a", "date": "Tue, 17 Dec 2024 21:32:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
1790	2024-12-17T21:32:06.313Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
1791	2024-12-17T21:32:06.315Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "933a9cfb-da8b-4135-98ae-4da1875d65bf", "date": "Tue, 17 Dec 2024 21:32:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
1792	2024-12-17T21:32:06.315Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
1793	2024-12-17T21:32:06.317Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
1794	2024-12-17T21:32:06.317Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 381.720561ms iteration = 12 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
1795	2024-12-17T21:32:06.346Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
1796	2024-12-17T21:32:06.380Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1797	2024-12-17T21:32:06.381Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1798	2024-12-17T21:32:06.383Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 1a3e1a79-c78b-4fb4-921b-c5e1e32e8b40, rack_id: 8216c8d1-8011-434d-a57a-2b1ce2e89ad8, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(d6c2617a-6b89-4779-bc70-3002fb0a7e5e) } rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0
1799	2024-12-17T21:32:06.383Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1800	2024-12-17T21:32:06.383Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1801	2024-12-17T21:32:06.393Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
1802	2024-12-17T21:32:06.394Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "c932f84f-40fd-41b7-a378-72c5b8534e07", "content-length": "134", "date": "Tue, 17 Dec 2024 21:32:06 GMT"} })
1803	2024-12-17T21:32:06.394Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
1804	2024-12-17T21:32:06.394Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = d6c2617a-6b89-4779-bc70-3002fb0a7e5e
1805	2024-12-17T21:32:06.394Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
1806	2024-12-17T21:32:06.394Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
1807	2024-12-17T21:32:06.394Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
1808	2024-12-17T21:32:06.394Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
1809	2024-12-17T21:32:06.394Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
1810	2024-12-17T21:32:06.394Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
1811	2024-12-17T21:32:06.395Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
1812	2024-12-17T21:32:06.395Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
1813	2024-12-17T21:32:06.395Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
1814	2024-12-17T21:32:06.395Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1815	2024-12-17T21:32:06.395Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1816	2024-12-17T21:32:06.397Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
1817	2024-12-17T21:32:06.397Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "411ef981-a6df-4483-ac18-4ad8cc209090", "content-length": "2", "date": "Tue, 17 Dec 2024 21:32:06 GMT"} })
1818	2024-12-17T21:32:06.397Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
1819	2024-12-17T21:32:06.397Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac9550e4-39cb-4791-ae25-f5ce9ffed6ba", "content-length": "51", "date": "Tue, 17 Dec 2024 21:32:06 GMT"} })
1820	2024-12-17T21:32:06.397Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0
1821	2024-12-17T21:32:06.397Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
1822	2024-12-17T21:32:06.412Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "03276b55-0a87-41a8-9d5e-810e9b90cc05", "date": "Tue, 17 Dec 2024 21:32:06 GMT"} })
1823	2024-12-17T21:32:06.412Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1824	2024-12-17T21:32:06.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1825	2024-12-17T21:32:06.412Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e025068-d9dd-4705-9ad4-d6b1ad3d7a14", "content-length": "691", "date": "Tue, 17 Dec 2024 21:32:06 GMT"} })
1826	2024-12-17T21:32:06.413Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
1827	2024-12-17T21:32:06.413Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
1828	2024-12-17T21:32:06.413Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
1829	2024-12-17T21:32:06.414Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 478.172312ms iteration = 19 status = {}
1830	2024-12-17T21:32:06.414Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1831	2024-12-17T21:32:06.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1832	2024-12-17T21:32:06.418Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1833	2024-12-17T21:32:06.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1834	2024-12-17T21:32:06.422Z	INFO	nexus (BackgroundTasks): VPC route manager sled d3278cd4-2587-4f0f-89dd-b4d4669a6ffe push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
1835	2024-12-17T21:32:06.422Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
1836	2024-12-17T21:32:06.423Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "0b2ad31a-c7c1-483d-afbb-e4b157f66e04", "date": "Tue, 17 Dec 2024 21:32:06 GMT"} })
1837	2024-12-17T21:32:06.424Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 488.221602ms iteration = 19 status = {}
1838	2024-12-17T21:32:06.509Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "50b33e94-1728-47f0-bd9b-c5b3394500b7", "date": "Tue, 17 Dec 2024 21:32:06 GMT"} })
1839	2024-12-17T21:32:06.509Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch = Switch0
1840	2024-12-17T21:32:06.509Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
1841	2024-12-17T21:32:06.509Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "afc13c0b-6444-42ff-a215-f4f3c1df439b", "content-length": "682", "date": "Tue, 17 Dec 2024 21:32:06 GMT"} })
1842	2024-12-17T21:32:06.509Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1843	2024-12-17T21:32:06.509Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1844	2024-12-17T21:32:06.512Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1845	2024-12-17T21:32:06.512Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1846	2024-12-17T21:32:06.515Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1847	2024-12-17T21:32:06.516Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1848	2024-12-17T21:32:06.518Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1849	2024-12-17T21:32:06.518Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1850	2024-12-17T21:32:06.525Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 589.591867ms iteration = 19 status = {}
1851	2024-12-17T21:32:10.407Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 314 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = 0a9a1f39-d2ef-4ca3-8f07-4c5d97477243 response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
1852	2024-12-17T21:32:15.935Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 56 reason = Timeout
1853	2024-12-17T21:32:15.935Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1854	2024-12-17T21:32:15.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1855	2024-12-17T21:32:15.936Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 56 reason = Timeout
1856	2024-12-17T21:32:15.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1857	2024-12-17T21:32:15.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1858	2024-12-17T21:32:15.937Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1859	2024-12-17T21:32:15.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1860	2024-12-17T21:32:15.940Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1861	2024-12-17T21:32:15.940Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1862	2024-12-17T21:32:15.942Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1863	2024-12-17T21:32:15.942Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1864	2024-12-17T21:32:15.946Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1865	2024-12-17T21:32:15.948Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
1866	2024-12-17T21:32:15.948Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1867	2024-12-17T21:32:15.948Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1868	2024-12-17T21:32:15.961Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
1869	2024-12-17T21:32:15.961Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.945072ms iteration = 56 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
1870	2024-12-17T21:32:15.968Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1871	2024-12-17T21:32:15.969Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1872	2024-12-17T21:32:15.970Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 34.022891ms iteration = 56 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
1873	2024-12-17T21:32:20.406Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 259 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = d0adf6f4-ab30-4a0f-9ebd-7dfa2e5a6a28 response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
1874	2024-12-17T21:32:25.394Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = bdc73366-ad45-4fca-a131-49fb6464009c roles = RoleSet { roles: {} } uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?sort_by=id_ascending
1875	2024-12-17T21:32:25.395Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = bdc73366-ad45-4fca-a131-49fb6464009c resource = Database result = Ok(()) uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?sort_by=id_ascending
1876	2024-12-17T21:32:25.397Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2341 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = bdc73366-ad45-4fca-a131-49fb6464009c response_code = 200 uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?sort_by=id_ascending
1877	2024-12-17T21:32:25.397Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 6cba563e-d7bf-46de-a9c6-28d24a03811e roles = RoleSet { roles: {} } uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMTkzZjYyMi00MGY1LTQyOGYtYmVhMy01MmNjNjdlNzhhMzkifX0%3D
1878	2024-12-17T21:32:25.397Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 6cba563e-d7bf-46de-a9c6-28d24a03811e resource = Database result = Ok(()) uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMTkzZjYyMi00MGY1LTQyOGYtYmVhMy01MmNjNjdlNzhhMzkifX0%3D
1879	2024-12-17T21:32:25.399Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2285 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 6cba563e-d7bf-46de-a9c6-28d24a03811e response_code = 200 uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMTkzZjYyMi00MGY1LTQyOGYtYmVhMy01MmNjNjdlNzhhMzkifX0%3D
1880	2024-12-17T21:32:25.935Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 57 reason = Timeout
1881	2024-12-17T21:32:25.935Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1882	2024-12-17T21:32:25.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1883	2024-12-17T21:32:25.936Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 57 reason = Timeout
1884	2024-12-17T21:32:25.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1885	2024-12-17T21:32:25.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1886	2024-12-17T21:32:25.937Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1887	2024-12-17T21:32:25.940Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1888	2024-12-17T21:32:25.940Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1889	2024-12-17T21:32:25.940Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1890	2024-12-17T21:32:25.941Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1891	2024-12-17T21:32:25.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1892	2024-12-17T21:32:25.943Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1893	2024-12-17T21:32:25.947Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
1894	2024-12-17T21:32:25.947Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1895	2024-12-17T21:32:25.948Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1896	2024-12-17T21:32:25.960Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
1897	2024-12-17T21:32:25.960Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.453272ms iteration = 57 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
1898	2024-12-17T21:32:25.971Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1899	2024-12-17T21:32:25.971Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1900	2024-12-17T21:32:25.999Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 63.165199ms iteration = 57 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
1901	2024-12-17T21:32:30.407Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 299 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = de4f32d6-c129-45a0-881e-1a740b75b0cd response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
1902	2024-12-17T21:32:35.935Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 58 reason = Timeout
1903	2024-12-17T21:32:35.935Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 20 reason = Timeout
1904	2024-12-17T21:32:35.935Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1905	2024-12-17T21:32:35.935Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 20 reason = Timeout
1906	2024-12-17T21:32:35.935Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1907	2024-12-17T21:32:35.935Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 20 reason = Timeout
1908	2024-12-17T21:32:35.935Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1909	2024-12-17T21:32:35.935Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 20 reason = Timeout
1910	2024-12-17T21:32:35.935Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 20 reason = Timeout
1911	2024-12-17T21:32:35.935Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 21 reason = Timeout
1912	2024-12-17T21:32:35.935Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
1913	2024-12-17T21:32:35.935Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
1914	2024-12-17T21:32:35.935Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1915	2024-12-17T21:32:35.935Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 20 reason = Timeout
1916	2024-12-17T21:32:35.935Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
1917	2024-12-17T21:32:35.935Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 20 reason = Timeout
1918	2024-12-17T21:32:35.935Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1919	2024-12-17T21:32:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1920	2024-12-17T21:32:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1921	2024-12-17T21:32:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1922	2024-12-17T21:32:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
1923	2024-12-17T21:32:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1924	2024-12-17T21:32:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1925	2024-12-17T21:32:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
1926	2024-12-17T21:32:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1927	2024-12-17T21:32:35.936Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 20 reason = Timeout
1928	2024-12-17T21:32:35.936Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 20 reason = Timeout
1929	2024-12-17T21:32:35.936Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
1930	2024-12-17T21:32:35.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
1931	2024-12-17T21:32:35.936Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 20 reason = Timeout
1932	2024-12-17T21:32:35.936Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 20 reason = Timeout
1933	2024-12-17T21:32:35.936Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 20 reason = Timeout
1934	2024-12-17T21:32:35.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
1935	2024-12-17T21:32:35.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
1936	2024-12-17T21:32:35.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
1937	2024-12-17T21:32:35.936Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 20 reason = Timeout
1938	2024-12-17T21:32:35.936Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 58 reason = Timeout
1939	2024-12-17T21:32:35.936Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
1940	2024-12-17T21:32:35.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1941	2024-12-17T21:32:35.936Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1942	2024-12-17T21:32:35.936Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 20 reason = Timeout
1943	2024-12-17T21:32:35.936Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
1944	2024-12-17T21:32:35.936Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
1945	2024-12-17T21:32:35.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1946	2024-12-17T21:32:35.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1947	2024-12-17T21:32:35.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
1948	2024-12-17T21:32:35.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
1949	2024-12-17T21:32:35.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
1950	2024-12-17T21:32:35.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
1951	2024-12-17T21:32:35.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
1952	2024-12-17T21:32:35.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1953	2024-12-17T21:32:35.937Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1954	2024-12-17T21:32:35.937Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1955	2024-12-17T21:32:35.937Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.812616ms iteration = 20 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
1956	2024-12-17T21:32:35.938Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1957	2024-12-17T21:32:35.939Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
1958	2024-12-17T21:32:35.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
1959	2024-12-17T21:32:35.940Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1814, tv_nsec: 72031735 } })
1960	2024-12-17T21:32:35.940Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
1961	2024-12-17T21:32:35.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
1962	2024-12-17T21:32:35.941Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1963	2024-12-17T21:32:35.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1964	2024-12-17T21:32:35.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
1965	2024-12-17T21:32:35.941Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1966	2024-12-17T21:32:35.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1967	2024-12-17T21:32:35.941Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1968	2024-12-17T21:32:35.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1969	2024-12-17T21:32:35.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1970	2024-12-17T21:32:35.943Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1971	2024-12-17T21:32:35.944Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1814, tv_nsec: 76018953 } })
1972	2024-12-17T21:32:35.944Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1973	2024-12-17T21:32:35.944Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1974	2024-12-17T21:32:35.944Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
1975	2024-12-17T21:32:35.944Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1976	2024-12-17T21:32:35.944Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 8.201272ms iteration = 20 status = {"errors":[],"records_set_to_done":[]}
1977	2024-12-17T21:32:35.946Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
1978	2024-12-17T21:32:35.946Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1979	2024-12-17T21:32:35.946Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
1980	2024-12-17T21:32:35.946Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1981	2024-12-17T21:32:35.946Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1982	2024-12-17T21:32:35.946Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1983	2024-12-17T21:32:35.946Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
1984	2024-12-17T21:32:35.946Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
1985	2024-12-17T21:32:35.946Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
1986	2024-12-17T21:32:35.947Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
1987	2024-12-17T21:32:35.947Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1988	2024-12-17T21:32:35.949Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
1989	2024-12-17T21:32:35.949Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 14.369809ms iteration = 20 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
1990	2024-12-17T21:32:35.949Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1991	2024-12-17T21:32:35.949Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1992	2024-12-17T21:32:35.950Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1993	2024-12-17T21:32:35.950Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
1994	2024-12-17T21:32:35.950Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
1995	2024-12-17T21:32:35.950Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
1996	2024-12-17T21:32:35.952Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1997	2024-12-17T21:32:35.952Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 16.097216ms iteration = 58 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
1998	2024-12-17T21:32:35.953Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1999	2024-12-17T21:32:35.953Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2000	2024-12-17T21:32:35.953Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2001	2024-12-17T21:32:35.953Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1814, tv_nsec: 85512270 } })
2002	2024-12-17T21:32:35.953Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
2003	2024-12-17T21:32:35.955Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2004	2024-12-17T21:32:35.955Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
2005	2024-12-17T21:32:35.955Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2006	2024-12-17T21:32:35.956Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
2007	2024-12-17T21:32:35.956Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2008	2024-12-17T21:32:35.956Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2009	2024-12-17T21:32:35.958Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2010	2024-12-17T21:32:35.958Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
2011	2024-12-17T21:32:35.958Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 21.542183ms iteration = 20 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
2012	2024-12-17T21:32:35.959Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2013	2024-12-17T21:32:35.959Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2014	2024-12-17T21:32:35.960Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2015	2024-12-17T21:32:35.960Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
2016	2024-12-17T21:32:35.960Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2017	2024-12-17T21:32:35.960Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2018	2024-12-17T21:32:35.962Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
2019	2024-12-17T21:32:35.962Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 25.705665ms iteration = 20 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
2020	2024-12-17T21:32:35.970Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
2021	2024-12-17T21:32:35.971Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 35.953123ms iteration = 58 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
2022	2024-12-17T21:32:35.973Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2023	2024-12-17T21:32:35.974Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2024	2024-12-17T21:32:35.976Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2025	2024-12-17T21:32:35.976Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2026	2024-12-17T21:32:35.987Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2027	2024-12-17T21:32:35.988Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2028	2024-12-17T21:32:35.991Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 54.575277ms iteration = 20 status = {"errors":[],"garbage_collect_requested":[]}
2029	2024-12-17T21:32:35.996Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
2030	2024-12-17T21:32:35.996Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
2031	2024-12-17T21:32:35.997Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2032	2024-12-17T21:32:35.998Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
2033	2024-12-17T21:32:35.998Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2034	2024-12-17T21:32:35.998Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2035	2024-12-17T21:32:36.014Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
2036	2024-12-17T21:32:36.014Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2037	2024-12-17T21:32:36.014Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b8f29e5f-5619-4cd5-8669-82a27f6ac8bf", "content-length": "26", "date": "Tue, 17 Dec 2024 21:32:35 GMT"} })
2038	2024-12-17T21:32:36.014Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2039	2024-12-17T21:32:36.017Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
2040	2024-12-17T21:32:36.021Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2041	2024-12-17T21:32:36.023Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: d3278cd4-2587-4f0f-89dd-b4d4669a6ffe, lookup_type: ById(d3278cd4-2587-4f0f-89dd-b4d4669a6ffe) } result = Ok(())
2042	2024-12-17T21:32:36.023Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2043	2024-12-17T21:32:36.024Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2044	2024-12-17T21:32:36.024Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 88.055113ms iteration = 20 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
2045	2024-12-17T21:32:36.038Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 102.170335ms iteration = 20 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
2046	2024-12-17T21:32:36.043Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 zone_address = fd00:1122:3344:101::2
2047	2024-12-17T21:32:36.043Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2048	2024-12-17T21:32:36.044Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7163e51e-cb10-4544-a730-b5bda0d61030", "content-length": "26", "date": "Tue, 17 Dec 2024 21:32:35 GMT"} })
2049	2024-12-17T21:32:36.044Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2050	2024-12-17T21:32:36.045Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
2051	2024-12-17T21:32:36.054Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
2052	2024-12-17T21:32:36.055Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2053	2024-12-17T21:32:36.055Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f9518c0-86dd-4263-afcc-261d7b4020b2", "content-length": "26", "date": "Tue, 17 Dec 2024 21:32:35 GMT"} })
2054	2024-12-17T21:32:36.055Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2055	2024-12-17T21:32:36.056Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
2056	2024-12-17T21:32:36.057Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2057	2024-12-17T21:32:36.057Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2058	2024-12-17T21:32:36.066Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 131.416857ms iteration = 21 status = {"physical_disks_added":0}
2059	2024-12-17T21:32:36.191Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 256.665246ms iteration = 20 status = {}
2060	2024-12-17T21:32:36.219Z	INFO	nexus (BackgroundTasks): VPC route manager sled d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
2061	2024-12-17T21:32:36.219Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
2062	2024-12-17T21:32:36.220Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd3a5772-b1e0-43b7-a0c8-baf9a38bb58f", "content-length": "372", "date": "Tue, 17 Dec 2024 21:32:36 GMT"} })
2063	2024-12-17T21:32:36.220Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2064	2024-12-17T21:32:36.220Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2065	2024-12-17T21:32:36.221Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 286.891184ms iteration = 20 status = 0
2066	2024-12-17T21:32:36.226Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
2067	2024-12-17T21:32:36.227Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "83c533f1-03d8-4de5-b71f-67569f99a4ec", "content-length": "1", "date": "Tue, 17 Dec 2024 21:32:35 GMT"} })
2068	2024-12-17T21:32:36.228Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
2069	2024-12-17T21:32:36.228Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
2070	2024-12-17T21:32:36.230Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 295.486126ms iteration = 20 status = null
2071	2024-12-17T21:32:36.258Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2072	2024-12-17T21:32:36.258Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2073	2024-12-17T21:32:36.264Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
2074	2024-12-17T21:32:36.264Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
2075	2024-12-17T21:32:36.265Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "4788ede0-1777-4d9a-9981-ebe9fc854c3b", "date": "Tue, 17 Dec 2024 21:32:36 GMT"} })
2076	2024-12-17T21:32:36.265Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2077	2024-12-17T21:32:36.265Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2078	2024-12-17T21:32:36.267Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2079	2024-12-17T21:32:36.267Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2080	2024-12-17T21:32:36.270Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2081	2024-12-17T21:32:36.270Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2082	2024-12-17T21:32:36.273Z	INFO	nexus (BackgroundTasks): VPC route manager sled d3278cd4-2587-4f0f-89dd-b4d4669a6ffe push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
2083	2024-12-17T21:32:36.273Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
2084	2024-12-17T21:32:36.274Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "d430bd89-ba74-4a2c-8118-f17e1ffe35eb", "date": "Tue, 17 Dec 2024 21:32:36 GMT"} })
2085	2024-12-17T21:32:36.274Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2086	2024-12-17T21:32:36.275Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2087	2024-12-17T21:32:36.276Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 339.604681ms iteration = 20 status = {}
2088	2024-12-17T21:32:36.276Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 1a3e1a79-c78b-4fb4-921b-c5e1e32e8b40, rack_id: 8216c8d1-8011-434d-a57a-2b1ce2e89ad8, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(d6c2617a-6b89-4779-bc70-3002fb0a7e5e) } rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0
2089	2024-12-17T21:32:36.276Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2090	2024-12-17T21:32:36.276Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2091	2024-12-17T21:32:36.286Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
2092	2024-12-17T21:32:36.286Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "775c50e7-7bde-4aa7-9638-545a0f2ba55c", "content-length": "134", "date": "Tue, 17 Dec 2024 21:32:35 GMT"} })
2093	2024-12-17T21:32:36.286Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
2094	2024-12-17T21:32:36.286Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = d6c2617a-6b89-4779-bc70-3002fb0a7e5e
2095	2024-12-17T21:32:36.287Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2096	2024-12-17T21:32:36.287Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
2097	2024-12-17T21:32:36.287Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
2098	2024-12-17T21:32:36.287Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2099	2024-12-17T21:32:36.287Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2100	2024-12-17T21:32:36.287Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2101	2024-12-17T21:32:36.287Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
2102	2024-12-17T21:32:36.287Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2103	2024-12-17T21:32:36.287Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
2104	2024-12-17T21:32:36.287Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
2105	2024-12-17T21:32:36.287Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2106	2024-12-17T21:32:36.287Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2107	2024-12-17T21:32:36.288Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
2108	2024-12-17T21:32:36.288Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2109	2024-12-17T21:32:36.289Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2110	2024-12-17T21:32:36.289Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
2111	2024-12-17T21:32:36.289Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "716da3c7-de44-44dd-a0d3-0024c35dcde6", "content-length": "2", "date": "Tue, 17 Dec 2024 21:32:35 GMT"} })
2112	2024-12-17T21:32:36.289Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
2113	2024-12-17T21:32:36.289Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "a0c79d0a-fb77-4496-969a-214e95b788be", "content-length": "51", "date": "Tue, 17 Dec 2024 21:32:35 GMT"} })
2114	2024-12-17T21:32:36.289Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0
2115	2024-12-17T21:32:36.289Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
2116	2024-12-17T21:32:36.290Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2117	2024-12-17T21:32:36.292Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
2118	2024-12-17T21:32:36.292Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2119	2024-12-17T21:32:36.292Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2120	2024-12-17T21:32:36.388Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
2121	2024-12-17T21:32:36.391Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "9587557c-6d9b-4eb6-8958-dfdd11bdbc91", "date": "Tue, 17 Dec 2024 21:32:36 GMT"} })
2122	2024-12-17T21:32:36.391Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch = Switch0
2123	2024-12-17T21:32:36.391Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
2124	2024-12-17T21:32:36.392Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "e737eb14-92d3-4bef-b044-adf0ead1a915", "content-length": "682", "date": "Tue, 17 Dec 2024 21:32:36 GMT"} })
2125	2024-12-17T21:32:36.392Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2126	2024-12-17T21:32:36.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2127	2024-12-17T21:32:36.392Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "0f31b99b-d504-453f-ab47-e4b479bbae0d", "content-length": "691", "date": "Tue, 17 Dec 2024 21:32:35 GMT"} })
2128	2024-12-17T21:32:36.392Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
2129	2024-12-17T21:32:36.392Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
2130	2024-12-17T21:32:36.392Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
2131	2024-12-17T21:32:36.394Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 459.904327ms iteration = 20 status = {}
2132	2024-12-17T21:32:36.396Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2133	2024-12-17T21:32:36.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2134	2024-12-17T21:32:36.400Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2135	2024-12-17T21:32:36.400Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2136	2024-12-17T21:32:36.402Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2137	2024-12-17T21:32:36.402Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2138	2024-12-17T21:32:36.409Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 474.529742ms iteration = 20 status = {}
2139	2024-12-17T21:32:40.407Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 292 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = 17b1d8f9-c03a-4116-899d-e67b6023143c response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
2140	2024-12-17T21:32:45.935Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 59 reason = Timeout
2141	2024-12-17T21:32:45.935Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2142	2024-12-17T21:32:45.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2143	2024-12-17T21:32:45.936Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 59 reason = Timeout
2144	2024-12-17T21:32:45.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2145	2024-12-17T21:32:45.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2146	2024-12-17T21:32:45.937Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2147	2024-12-17T21:32:45.940Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2148	2024-12-17T21:32:45.940Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2149	2024-12-17T21:32:45.940Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2150	2024-12-17T21:32:45.942Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2151	2024-12-17T21:32:45.942Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2152	2024-12-17T21:32:45.969Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2153	2024-12-17T21:32:45.969Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2154	2024-12-17T21:32:45.970Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 34.73839ms iteration = 59 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
2155	2024-12-17T21:32:45.974Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2156	2024-12-17T21:32:45.977Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
2157	2024-12-17T21:32:45.977Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2158	2024-12-17T21:32:45.977Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2159	2024-12-17T21:32:45.988Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
2160	2024-12-17T21:32:45.988Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 52.583432ms iteration = 59 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
2161	2024-12-17T21:32:50.406Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 295 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = 9bc80916-d7bc-46e4-97af-1b52cd14db25 response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
2162	2024-12-17T21:32:55.394Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 92a60a2f-916d-4dae-bae6-492000eb8e73 roles = RoleSet { roles: {} } uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?sort_by=id_ascending
2163	2024-12-17T21:32:55.395Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 92a60a2f-916d-4dae-bae6-492000eb8e73 resource = Database result = Ok(()) uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?sort_by=id_ascending
2164	2024-12-17T21:32:55.397Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3111 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 92a60a2f-916d-4dae-bae6-492000eb8e73 response_code = 200 uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?sort_by=id_ascending
2165	2024-12-17T21:32:55.398Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 4699dc35-d750-4d30-8700-05eb93fc8cc9 roles = RoleSet { roles: {} } uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMTkzZjYyMi00MGY1LTQyOGYtYmVhMy01MmNjNjdlNzhhMzkifX0%3D
2166	2024-12-17T21:32:55.398Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 4699dc35-d750-4d30-8700-05eb93fc8cc9 resource = Database result = Ok(()) uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMTkzZjYyMi00MGY1LTQyOGYtYmVhMy01MmNjNjdlNzhhMzkifX0%3D
2167	2024-12-17T21:32:55.400Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2130 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 4699dc35-d750-4d30-8700-05eb93fc8cc9 response_code = 200 uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMTkzZjYyMi00MGY1LTQyOGYtYmVhMy01MmNjNjdlNzhhMzkifX0%3D
2168	2024-12-17T21:32:55.935Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 60 reason = Timeout
2169	2024-12-17T21:32:55.935Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2170	2024-12-17T21:32:55.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2171	2024-12-17T21:32:55.935Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 60 reason = Timeout
2172	2024-12-17T21:32:55.935Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2173	2024-12-17T21:32:55.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2174	2024-12-17T21:32:55.938Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2175	2024-12-17T21:32:55.938Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2176	2024-12-17T21:32:55.939Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.810553ms iteration = 60 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
2177	2024-12-17T21:32:55.964Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2178	2024-12-17T21:32:55.966Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2179	2024-12-17T21:32:55.966Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2180	2024-12-17T21:32:55.967Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2181	2024-12-17T21:32:55.968Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2182	2024-12-17T21:32:55.969Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2183	2024-12-17T21:32:55.970Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2184	2024-12-17T21:32:55.972Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
2185	2024-12-17T21:32:55.972Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2186	2024-12-17T21:32:55.972Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2187	2024-12-17T21:32:55.984Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
2188	2024-12-17T21:32:55.984Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 49.091238ms iteration = 60 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
2189	2024-12-17T21:33:00.407Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 301 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = 1dbf1c57-5700-4f57-a4c8-5c5a210bc9e8 response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
2190	2024-12-17T21:33:05.935Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 11 reason = Timeout
2191	2024-12-17T21:33:05.935Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 12 reason = Timeout
2192	2024-12-17T21:33:05.935Z	WARN	nexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5
2193	2024-12-17T21:33:05.935Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 21 reason = Timeout
2194	2024-12-17T21:33:05.935Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 19.951\u{b5}s iteration = 12 status = {"enabled":false,"target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5"}
2195	2024-12-17T21:33:05.935Z	DEBG	nexus (BackgroundTasks): activating background_task = inventory_collection iteration = 2 reason = Timeout
2196	2024-12-17T21:33:05.935Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 21 reason = Timeout
2197	2024-12-17T21:33:05.935Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2198	2024-12-17T21:33:05.935Z	DEBG	nexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
2199	2024-12-17T21:33:05.935Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2200	2024-12-17T21:33:05.935Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 13 reason = Timeout
2201	2024-12-17T21:33:05.935Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 11 reason = Timeout
2202	2024-12-17T21:33:05.935Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 22 reason = Timeout
2203	2024-12-17T21:33:05.935Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
2204	2024-12-17T21:33:05.935Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2205	2024-12-17T21:33:05.935Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2206	2024-12-17T21:33:05.935Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 21 reason = Timeout
2207	2024-12-17T21:33:05.935Z	DEBG	nexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 11 reason = Timeout
2208	2024-12-17T21:33:05.935Z	INFO	nexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
2209	2024-12-17T21:33:05.935Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 21 reason = Timeout
2210	2024-12-17T21:33:05.935Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
2211	2024-12-17T21:33:05.935Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2212	2024-12-17T21:33:05.935Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
2213	2024-12-17T21:33:05.935Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 21 reason = Timeout
2214	2024-12-17T21:33:05.935Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
2215	2024-12-17T21:33:05.935Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 21 reason = Timeout
2216	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 13 reason = Timeout
2217	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 11 reason = Timeout
2218	2024-12-17T21:33:05.936Z	INFO	nexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-17 21:23:05.935861325 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
2219	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
2220	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 11 reason = Timeout
2221	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = external_endpoints iteration = 11 reason = Timeout
2222	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2223	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 11 reason = Timeout
2224	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
2225	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2226	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 21 reason = Timeout
2227	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
2228	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 61 reason = Timeout
2229	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2230	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 21 reason = Timeout
2231	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
2232	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 21 reason = Timeout
2233	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
2234	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 11 reason = Timeout
2235	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2236	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
2237	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 21 reason = Timeout
2238	2024-12-17T21:33:05.936Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
2239	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 21 reason = Timeout
2240	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2241	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = saga_recovery iteration = 2 reason = Timeout
2242	2024-12-17T21:33:05.936Z	DEBG	nexus (SagaRecoverer): listing candidate sagas for recovery
2243	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 11 reason = Timeout
2244	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
2245	2024-12-17T21:33:05.936Z	DEBG	nexus (SagaRecoverer): roles roles = RoleSet { roles: {} }
2246	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
2247	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 11 reason = Timeout
2248	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
2249	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 61 reason = Timeout
2250	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2251	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 21 reason = Timeout
2252	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 21 reason = Timeout
2253	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
2254	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2255	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 21 reason = Timeout
2256	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
2257	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 3 reason = Timeout
2258	2024-12-17T21:33:05.936Z	DEBG	nexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
2259	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2260	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2261	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2262	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2263	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2264	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
2265	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
2266	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
2267	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2268	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
2269	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
2270	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2271	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
2272	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
2273	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2274	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
2275	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
2276	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
2277	2024-12-17T21:33:05.936Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("5dff2793-3ce8-4251-ba71-1a446966cbf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a761a47e-08b9-41c4-9bd8-8d77651dfa62.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a761a47e-08b9-41c4-9bd8-8d77651dfa62.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1844, tv_nsec: 68831969 } })
2278	2024-12-17T21:33:05.936Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0005c310-624c-4312-b0fd-35f571746851.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6ac4d5e8-fcd9-4aa9-aed5-fb3dc293aac2.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9b804364-993d-47ba-81a1-d0ef32f96529.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("9b804364-993d-47ba-81a1-d0ef32f96529.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1844, tv_nsec: 68842139 } })
2279	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2280	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
2281	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2282	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
2283	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2284	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2285	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
2286	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2287	2024-12-17T21:33:05.936Z	DEBG	nexus (SagaRecoverer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a, .. }) resource = Database result = Ok(())
2288	2024-12-17T21:33:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
2289	2024-12-17T21:33:05.937Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
2290	2024-12-17T21:33:05.937Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.262314ms iteration = 11 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
2291	2024-12-17T21:33:05.937Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
2292	2024-12-17T21:33:05.937Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.579932ms iteration = 11 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
2293	2024-12-17T21:33:05.937Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2294	2024-12-17T21:33:05.938Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2295	2024-12-17T21:33:05.940Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2296	2024-12-17T21:33:05.940Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.483201ms iteration = 21 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
2297	2024-12-17T21:33:05.940Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2298	2024-12-17T21:33:05.940Z	DEBG	nexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2299	2024-12-17T21:33:05.942Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1844, tv_nsec: 74323227 } })
2300	2024-12-17T21:33:05.942Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
2301	2024-12-17T21:33:05.942Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
2302	2024-12-17T21:33:05.942Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2303	2024-12-17T21:33:05.943Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2304	2024-12-17T21:33:05.943Z	DEBG	nexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
2305	2024-12-17T21:33:05.943Z	DEBG	nexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
2306	2024-12-17T21:33:05.943Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
2307	2024-12-17T21:33:05.944Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2308	2024-12-17T21:33:05.944Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2309	2024-12-17T21:33:05.944Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2310	2024-12-17T21:33:05.945Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2311	2024-12-17T21:33:05.945Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2312	2024-12-17T21:33:05.947Z	INFO	nexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-17 21:23:05.935861325 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
2313	2024-12-17T21:33:05.947Z	DEBG	nexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 11.580863ms iteration = 11 status = {"expiration":"2024-12-17T21:23:05.935861325Z","pruned":[]}
2314	2024-12-17T21:33:05.947Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2315	2024-12-17T21:33:05.948Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2316	2024-12-17T21:33:05.949Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1844, tv_nsec: 81862361 } })
2317	2024-12-17T21:33:05.950Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
2318	2024-12-17T21:33:05.951Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
2319	2024-12-17T21:33:05.951Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2320	2024-12-17T21:33:05.951Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2321	2024-12-17T21:33:05.951Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
2322	2024-12-17T21:33:05.951Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2323	2024-12-17T21:33:05.952Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2324	2024-12-17T21:33:05.961Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2325	2024-12-17T21:33:05.962Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2326	2024-12-17T21:33:05.962Z	DEBG	nexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 26.638411ms iteration = 11 status = {"errors":[],"found_port_ok":[]}
2327	2024-12-17T21:33:05.966Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1844, tv_nsec: 98400199 } })
2328	2024-12-17T21:33:05.966Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
2329	2024-12-17T21:33:05.966Z	DEBG	nexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1591a6b7-7450-4c19-b5bb-557953732255, 0), (eab14eaa-1421-44b1-86a6-25b2c047c0fc, 0), (20bec956-d9be-49e4-ad72-2c02c48032bf, 0)]
2330	2024-12-17T21:33:05.967Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1844, tv_nsec: 99346644 } })
2331	2024-12-17T21:33:05.968Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2332	2024-12-17T21:33:05.969Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2333	2024-12-17T21:33:05.970Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2334	2024-12-17T21:33:05.971Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2335	2024-12-17T21:33:05.990Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 54.801193ms iteration = 21 status = {"errors":[],"records_set_to_done":[]}
2336	2024-12-17T21:33:05.993Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2337	2024-12-17T21:33:05.995Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
2338	2024-12-17T21:33:05.995Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2339	2024-12-17T21:33:05.995Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2340	2024-12-17T21:33:05.997Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
2341	2024-12-17T21:33:05.997Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2342	2024-12-17T21:33:05.997Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2343	2024-12-17T21:33:05.997Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
2344	2024-12-17T21:33:05.998Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2345	2024-12-17T21:33:05.998Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2346	2024-12-17T21:33:05.998Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
2347	2024-12-17T21:33:05.998Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2348	2024-12-17T21:33:05.998Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
2349	2024-12-17T21:33:05.999Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2350	2024-12-17T21:33:05.999Z	INFO	nexus (SagaRecoverer): listed in-progress sagas count = 0 file = nexus/src/app/background/tasks/saga_recovery.rs:438
2351	2024-12-17T21:33:05.999Z	DEBG	nexus (BackgroundTasks): activation complete background_task = saga_recovery elapsed = 63.721494ms iteration = 2 status = {"last_pass":{"Success":{"nfailed":0,"nfound":0,"nrecovered":0,"nremoved":0,"nskipped":0}},"ntotal_failures":0,"ntotal_finished":0,"ntotal_recovered":0,"ntotal_sec_errors_bad_state":0,"ntotal_sec_errors_missing":0,"ntotal_started":0,"recent_failures":[],"recent_recoveries":[]}
2352	2024-12-17T21:33:06.000Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
2353	2024-12-17T21:33:06.000Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2354	2024-12-17T21:33:06.000Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2355	2024-12-17T21:33:06.001Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2356	2024-12-17T21:33:06.001Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2357	2024-12-17T21:33:06.001Z	DEBG	nexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
2358	2024-12-17T21:33:06.001Z	INFO	nexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
2359	2024-12-17T21:33:06.001Z	DEBG	nexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 65.743989ms iteration = 11 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
2360	2024-12-17T21:33:06.002Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2361	2024-12-17T21:33:06.002Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2362	2024-12-17T21:33:06.003Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2363	2024-12-17T21:33:06.003Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2364	2024-12-17T21:33:06.003Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2365	2024-12-17T21:33:06.004Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2366	2024-12-17T21:33:06.004Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2367	2024-12-17T21:33:06.004Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
2368	2024-12-17T21:33:06.004Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 68.500434ms iteration = 21 status = {"errors":[],"garbage_collect_requested":[]}
2369	2024-12-17T21:33:06.004Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2370	2024-12-17T21:33:06.004Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2371	2024-12-17T21:33:06.004Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
2372	2024-12-17T21:33:06.005Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2373	2024-12-17T21:33:06.005Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
2374	2024-12-17T21:33:06.005Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
2375	2024-12-17T21:33:06.005Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
2376	2024-12-17T21:33:06.006Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
2377	2024-12-17T21:33:06.006Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
2378	2024-12-17T21:33:06.006Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2379	2024-12-17T21:33:06.006Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2380	2024-12-17T21:33:06.006Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
2381	2024-12-17T21:33:06.006Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2382	2024-12-17T21:33:06.007Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2383	2024-12-17T21:33:06.008Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2384	2024-12-17T21:33:06.009Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2385	2024-12-17T21:33:06.010Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2386	2024-12-17T21:33:06.010Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2387	2024-12-17T21:33:06.010Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2388	2024-12-17T21:33:06.012Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2389	2024-12-17T21:33:06.013Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
2390	2024-12-17T21:33:06.013Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2391	2024-12-17T21:33:06.014Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2392	2024-12-17T21:33:06.019Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2393	2024-12-17T21:33:06.020Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2394	2024-12-17T21:33:06.029Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2395	2024-12-17T21:33:06.032Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
2396	2024-12-17T21:33:06.032Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2397	2024-12-17T21:33:06.032Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2398	2024-12-17T21:33:06.041Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
2399	2024-12-17T21:33:06.041Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
2400	2024-12-17T21:33:06.043Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2401	2024-12-17T21:33:06.043Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2402	2024-12-17T21:33:06.046Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2403	2024-12-17T21:33:06.046Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2404	2024-12-17T21:33:06.050Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
2405	2024-12-17T21:33:06.050Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 114.405716ms iteration = 61 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
2406	2024-12-17T21:33:06.052Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 116.202375ms iteration = 21 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
2407	2024-12-17T21:33:06.053Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2408	2024-12-17T21:33:06.053Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
2409	2024-12-17T21:33:06.053Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2410	2024-12-17T21:33:06.054Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
2411	2024-12-17T21:33:06.060Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
2412	2024-12-17T21:33:06.060Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
2413	2024-12-17T21:33:06.060Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
2414	2024-12-17T21:33:06.060Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2415	2024-12-17T21:33:06.060Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2416	2024-12-17T21:33:06.065Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2417	2024-12-17T21:33:06.065Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2418	2024-12-17T21:33:06.066Z	DEBG	nexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
2419	2024-12-17T21:33:06.066Z	INFO	nexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
2420	2024-12-17T21:33:06.066Z	DEBG	nexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 130.636976ms iteration = 11 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
2421	2024-12-17T21:33:06.068Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
2422	2024-12-17T21:33:06.068Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
2423	2024-12-17T21:33:06.068Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
2424	2024-12-17T21:33:06.068Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2425	2024-12-17T21:33:06.068Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2426	2024-12-17T21:33:06.069Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 133.652767ms iteration = 21 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
2427	2024-12-17T21:33:06.070Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2428	2024-12-17T21:33:06.072Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2429	2024-12-17T21:33:06.072Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2430	2024-12-17T21:33:06.073Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2431	2024-12-17T21:33:06.074Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2432	2024-12-17T21:33:06.074Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2433	2024-12-17T21:33:06.074Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2434	2024-12-17T21:33:06.074Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2435	2024-12-17T21:33:06.074Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2436	2024-12-17T21:33:06.076Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
2437	2024-12-17T21:33:06.076Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2438	2024-12-17T21:33:06.076Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2439	2024-12-17T21:33:06.076Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 140.631846ms iteration = 61 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
2440	2024-12-17T21:33:06.077Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2441	2024-12-17T21:33:06.077Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2442	2024-12-17T21:33:06.081Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
2443	2024-12-17T21:33:06.081Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 145.630111ms iteration = 21 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
2444	2024-12-17T21:33:06.091Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2445	2024-12-17T21:33:06.095Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: d3278cd4-2587-4f0f-89dd-b4d4669a6ffe, lookup_type: ById(d3278cd4-2587-4f0f-89dd-b4d4669a6ffe) } result = Ok(())
2446	2024-12-17T21:33:06.095Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2447	2024-12-17T21:33:06.095Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2448	2024-12-17T21:33:06.099Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 163.33878ms iteration = 21 status = 0
2449	2024-12-17T21:33:06.101Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 165.650703ms iteration = 21 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
2450	2024-12-17T21:33:06.107Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2451	2024-12-17T21:33:06.109Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2452	2024-12-17T21:33:06.109Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
2453	2024-12-17T21:33:06.109Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 173.105985ms iteration = 21 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
2454	2024-12-17T21:33:06.110Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2455	2024-12-17T21:33:06.110Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2456	2024-12-17T21:33:06.110Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2457	2024-12-17T21:33:06.118Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2458	2024-12-17T21:33:06.119Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2459	2024-12-17T21:33:06.124Z	DEBG	nexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-17T21:23:03.821125Z, time_modified: 2024-12-17T21:23:03.821125Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 81e78e1e-e55a-44f9-abdc-c9c916e1d9fd, db_silo: Silo { identity: SiloIdentity { id: 81e78e1e-e55a-44f9-abdc-c9c916e1d9fd, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-17T21:23:49.443930Z, time_modified: 2024-12-17T21:23:49.443930Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e275643b21ab02733b9616c9cd3b0193e27c7f6294b7d310d6f1726c72768e50" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 81e78e1e-e55a-44f9-abdc-c9c916e1d9fd, db_silo: Silo { identity: SiloIdentity { id: 81e78e1e-e55a-44f9-abdc-c9c916e1d9fd, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-17T21:23:49.443930Z, time_modified: 2024-12-17T21:23:49.443930Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e275643b21ab02733b9616c9cd3b0193e27c7f6294b7d310d6f1726c72768e50" }] }) }
2460	2024-12-17T21:33:06.124Z	DEBG	nexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 188.47194ms iteration = 11 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"81e78e1e-e55a-44f9-abdc-c9c916e1d9fd","tls_certs":["e275643b21ab02733b9616c9cd3b0193e27c7f6294b7d310d6f1726c72768e50"]}},"default_endpoint":{"silo_id":"81e78e1e-e55a-44f9-abdc-c9c916e1d9fd","tls_certs":["e275643b21ab02733b9616c9cd3b0193e27c7f6294b7d310d6f1726c72768e50"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
2461	2024-12-17T21:33:06.125Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
2462	2024-12-17T21:33:06.125Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 574f0c35-b8f2-48fe-831c-78afc4a62aaa dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
2463	2024-12-17T21:33:06.125Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2464	2024-12-17T21:33:06.125Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2465	2024-12-17T21:33:06.127Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2466	2024-12-17T21:33:06.130Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2467	2024-12-17T21:33:06.130Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2468	2024-12-17T21:33:06.131Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2469	2024-12-17T21:33:06.134Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 574f0c35-b8f2-48fe-831c-78afc4a62aaa dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
2470	2024-12-17T21:33:06.134Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
2471	2024-12-17T21:33:06.134Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-17 21:23:48.446255 UTC dns_group = internal generation = 1
2472	2024-12-17T21:33:06.134Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 198.348638ms iteration = 11 status = {"generation":1}
2473	2024-12-17T21:33:06.155Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
2474	2024-12-17T21:33:06.155Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = ea6acdde-77d4-4bf6-b578-d9a0a29ac37b dns_zone_name = oxide.test found_so_far = 0 version = 2
2475	2024-12-17T21:33:06.155Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2476	2024-12-17T21:33:06.156Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2477	2024-12-17T21:33:06.158Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
2478	2024-12-17T21:33:06.158Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2479	2024-12-17T21:33:06.158Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4cdc3e10-4916-4516-9805-9406527235a2", "content-length": "26", "date": "Tue, 17 Dec 2024 21:33:05 GMT"} })
2480	2024-12-17T21:33:06.158Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2481	2024-12-17T21:33:06.160Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2482	2024-12-17T21:33:06.161Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 225.639816ms iteration = 22 status = {"physical_disks_added":0}
2483	2024-12-17T21:33:06.161Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
2484	2024-12-17T21:33:06.164Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
2485	2024-12-17T21:33:06.164Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2486	2024-12-17T21:33:06.164Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2487	2024-12-17T21:33:06.173Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 zone_address = fd00:1122:3344:101::2
2488	2024-12-17T21:33:06.173Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2489	2024-12-17T21:33:06.173Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "002a6b56-852f-4f1d-80ac-3ef6fc0745c5", "content-length": "26", "date": "Tue, 17 Dec 2024 21:33:05 GMT"} })
2490	2024-12-17T21:33:06.173Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2491	2024-12-17T21:33:06.175Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
2492	2024-12-17T21:33:06.179Z	WARN	nexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
2493	2024-12-17T21:33:06.179Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 243.381296ms iteration = 11 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
2494	2024-12-17T21:33:06.194Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2495	2024-12-17T21:33:06.194Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2496	2024-12-17T21:33:06.197Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
2497	2024-12-17T21:33:06.287Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
2498	2024-12-17T21:33:06.321Z	INFO	nexus (BackgroundTasks): VPC route manager sled d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
2499	2024-12-17T21:33:06.321Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
2500	2024-12-17T21:33:06.321Z	DEBG	nexus (BackgroundTasks): begin collection background_task = inventory_collection
2501	2024-12-17T21:33:06.321Z	DEBG	nexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
2502	2024-12-17T21:33:06.321Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
2503	2024-12-17T21:33:06.322Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "cbcad82e-48a8-4b08-90f8-2806b9383c24", "content-length": "476", "date": "Tue, 17 Dec 2024 21:33:05 GMT"} })
2504	2024-12-17T21:33:06.322Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
2505	2024-12-17T21:33:06.322Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d478561-0f85-4681-a10e-136987e4d7b9", "content-length": "734", "date": "Tue, 17 Dec 2024 21:33:05 GMT"} })
2506	2024-12-17T21:33:06.322Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
2507	2024-12-17T21:33:06.322Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ccc3ef75-dbe2-4a7a-8f2a-176d4cfc3014", "content-length": "111", "date": "Tue, 17 Dec 2024 21:33:05 GMT"} })
2508	2024-12-17T21:33:06.322Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
2509	2024-12-17T21:33:06.323Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "791ffb4f-b5ee-49c3-996a-a82c8c31d1c6", "content-length": "111", "date": "Tue, 17 Dec 2024 21:33:05 GMT"} })
2510	2024-12-17T21:33:06.323Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
2511	2024-12-17T21:33:06.323Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "99dff2c6-a1ea-4ffc-a723-a4003f811853", "content-length": "167", "date": "Tue, 17 Dec 2024 21:33:05 GMT"} })
2512	2024-12-17T21:33:06.323Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
2513	2024-12-17T21:33:06.323Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "95a65cf0-3766-4617-be8c-0d2342f5f4b5", "content-length": "167", "date": "Tue, 17 Dec 2024 21:33:05 GMT"} })
2514	2024-12-17T21:33:06.323Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
2515	2024-12-17T21:33:06.323Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2516	2024-12-17T21:33:06.324Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "99771427-c3d3-4556-b2cf-0059e0b3357b", "content-length": "175", "date": "Tue, 17 Dec 2024 21:33:05 GMT"} })
2517	2024-12-17T21:33:06.324Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2518	2024-12-17T21:33:06.324Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
2519	2024-12-17T21:33:06.324Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6bc90199-f971-4fc9-b00d-80541b5520c9", "content-length": "175", "date": "Tue, 17 Dec 2024 21:33:05 GMT"} })
2520	2024-12-17T21:33:06.324Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
2521	2024-12-17T21:33:06.324Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "88084dcf-7786-4cd4-aa65-c980aa96887d", "content-length": "702", "date": "Tue, 17 Dec 2024 21:33:05 GMT"} })
2522	2024-12-17T21:33:06.324Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
2523	2024-12-17T21:33:06.324Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "43a82a59-6d23-465a-9bfe-7b5b624265ab", "content-length": "727", "date": "Tue, 17 Dec 2024 21:33:05 GMT"} })
2524	2024-12-17T21:33:06.324Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
2525	2024-12-17T21:33:06.324Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e75147b6-cafd-44d3-8022-301cbc05995a", "content-length": "729", "date": "Tue, 17 Dec 2024 21:33:05 GMT"} })
2526	2024-12-17T21:33:06.324Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
2527	2024-12-17T21:33:06.324Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8d8eecc6-4dd7-49b3-ba1c-ea2b8b9d0848", "content-length": "728", "date": "Tue, 17 Dec 2024 21:33:05 GMT"} })
2528	2024-12-17T21:33:06.324Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
2529	2024-12-17T21:33:06.325Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "799d1b95-22bc-4749-8041-13390e4b4064", "content-length": "722", "date": "Tue, 17 Dec 2024 21:33:05 GMT"} })
2530	2024-12-17T21:33:06.325Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
2531	2024-12-17T21:33:06.325Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae927a9e-ae29-43b6-bdaa-fc416dbe2a82", "content-length": "372", "date": "Tue, 17 Dec 2024 21:33:06 GMT"} })
2532	2024-12-17T21:33:06.325Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2533	2024-12-17T21:33:06.325Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d17003e1-9b3e-4f29-b6d1-6b692515a7e8", "content-length": "109", "date": "Tue, 17 Dec 2024 21:33:05 GMT"} })
2534	2024-12-17T21:33:06.325Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
2535	2024-12-17T21:33:06.325Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2536	2024-12-17T21:33:06.325Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2270e01-6cba-4b9b-8e6f-20ef1e2e948d", "content-length": "109", "date": "Tue, 17 Dec 2024 21:33:05 GMT"} })
2537	2024-12-17T21:33:06.325Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
2538	2024-12-17T21:33:06.326Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b663a03-41ba-41d2-80e6-92d29722ec1d", "content-length": "169", "date": "Tue, 17 Dec 2024 21:33:05 GMT"} })
2539	2024-12-17T21:33:06.326Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
2540	2024-12-17T21:33:06.326Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0c4ba2f0-abb9-45f5-a8f8-95e86214192f", "content-length": "169", "date": "Tue, 17 Dec 2024 21:33:05 GMT"} })
2541	2024-12-17T21:33:06.326Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
2542	2024-12-17T21:33:06.326Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5be8be37-d280-4f20-8049-1ec49a436d75", "content-length": "178", "date": "Tue, 17 Dec 2024 21:33:05 GMT"} })
2543	2024-12-17T21:33:06.326Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
2544	2024-12-17T21:33:06.327Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d95a3430-313f-48db-9b5b-7ad87ce47d02", "content-length": "178", "date": "Tue, 17 Dec 2024 21:33:05 GMT"} })
2545	2024-12-17T21:33:06.327Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
2546	2024-12-17T21:33:06.327Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2547	2024-12-17T21:33:06.327Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2548	2024-12-17T21:33:06.327Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
2549	2024-12-17T21:33:06.327Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2550	2024-12-17T21:33:06.327Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "88db1708-6326-4474-8046-6f27113903a4", "content-length": "702", "date": "Tue, 17 Dec 2024 21:33:05 GMT"} })
2551	2024-12-17T21:33:06.327Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
2552	2024-12-17T21:33:06.327Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2553	2024-12-17T21:33:06.327Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "439c2798-bea8-4174-8092-094677926df1", "content-length": "727", "date": "Tue, 17 Dec 2024 21:33:05 GMT"} })
2554	2024-12-17T21:33:06.327Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
2555	2024-12-17T21:33:06.327Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2556	2024-12-17T21:33:06.327Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
2557	2024-12-17T21:33:06.327Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2558	2024-12-17T21:33:06.327Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a4215ade-3104-4879-b102-8e30a9dd6eec", "content-length": "729", "date": "Tue, 17 Dec 2024 21:33:06 GMT"} })
2559	2024-12-17T21:33:06.327Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
2560	2024-12-17T21:33:06.327Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "dca72ad4-0d82-4dd5-bf34-df212417d96e", "content-length": "26", "date": "Tue, 17 Dec 2024 21:33:06 GMT"} })
2561	2024-12-17T21:33:06.327Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2562	2024-12-17T21:33:06.327Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "25b10e1e-a3c4-49f6-9dab-e28c287e7bb6", "content-length": "728", "date": "Tue, 17 Dec 2024 21:33:06 GMT"} })
2563	2024-12-17T21:33:06.328Z	DEBG	nexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
2564	2024-12-17T21:33:06.328Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
2565	2024-12-17T21:33:06.329Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
2566	2024-12-17T21:33:06.329Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = ea6acdde-77d4-4bf6-b578-d9a0a29ac37b dns_zone_name = oxide.test found_so_far = 1 version = 2
2567	2024-12-17T21:33:06.329Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
2568	2024-12-17T21:33:06.329Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-17 21:23:49.491999 UTC dns_group = external generation = 2
2569	2024-12-17T21:33:06.329Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 393.778427ms iteration = 11 status = {"generation":2}
2570	2024-12-17T21:33:06.329Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2571	2024-12-17T21:33:06.330Z	DEBG	nexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2572	2024-12-17T21:33:06.332Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
2573	2024-12-17T21:33:06.332Z	DEBG	nexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
2574	2024-12-17T21:33:06.332Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
2575	2024-12-17T21:33:06.332Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
2576	2024-12-17T21:33:06.332Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2577	2024-12-17T21:33:06.333Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2578	2024-12-17T21:33:06.333Z	DEBG	nexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2579	2024-12-17T21:33:06.335Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
2580	2024-12-17T21:33:06.335Z	DEBG	nexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
2581	2024-12-17T21:33:06.335Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
2582	2024-12-17T21:33:06.355Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2583	2024-12-17T21:33:06.355Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2584	2024-12-17T21:33:06.357Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2585	2024-12-17T21:33:06.358Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2586	2024-12-17T21:33:06.359Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2587	2024-12-17T21:33:06.363Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
2588	2024-12-17T21:33:06.363Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2589	2024-12-17T21:33:06.363Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2590	2024-12-17T21:33:06.365Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 429.561255ms iteration = 21 status = {}
2591	2024-12-17T21:33:06.366Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2592	2024-12-17T21:33:06.366Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2593	2024-12-17T21:33:06.368Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2594	2024-12-17T21:33:06.368Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2595	2024-12-17T21:33:06.372Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2596	2024-12-17T21:33:06.372Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2597	2024-12-17T21:33:06.378Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2598	2024-12-17T21:33:06.378Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2599	2024-12-17T21:33:06.380Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2600	2024-12-17T21:33:06.380Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2601	2024-12-17T21:33:06.382Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
2602	2024-12-17T21:33:06.382Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
2603	2024-12-17T21:33:06.382Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "d03c12b5-afea-485b-a6a2-85f72c56277b", "date": "Tue, 17 Dec 2024 21:33:06 GMT"} })
2604	2024-12-17T21:33:06.382Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2605	2024-12-17T21:33:06.382Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2606	2024-12-17T21:33:06.385Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2607	2024-12-17T21:33:06.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2608	2024-12-17T21:33:06.389Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2609	2024-12-17T21:33:06.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2610	2024-12-17T21:33:06.394Z	INFO	nexus (BackgroundTasks): VPC route manager sled d3278cd4-2587-4f0f-89dd-b4d4669a6ffe push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
2611	2024-12-17T21:33:06.394Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
2612	2024-12-17T21:33:06.395Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "111c06be-e1d4-4fc2-abf7-400ccaf178e4", "date": "Tue, 17 Dec 2024 21:33:06 GMT"} })
2613	2024-12-17T21:33:06.396Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 460.594305ms iteration = 21 status = {}
2614	2024-12-17T21:33:06.419Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2615	2024-12-17T21:33:06.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2616	2024-12-17T21:33:06.453Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
2617	2024-12-17T21:33:06.489Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
2618	2024-12-17T21:33:06.489Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "d23cb603-a266-4399-b30b-f2fee33bb9a0", "content-length": "1", "date": "Tue, 17 Dec 2024 21:33:05 GMT"} })
2619	2024-12-17T21:33:06.491Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
2620	2024-12-17T21:33:06.491Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
2621	2024-12-17T21:33:06.586Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
2622	2024-12-17T21:33:06.587Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c6441a6-3350-43ee-9e63-7bba229295d9", "content-length": "691", "date": "Tue, 17 Dec 2024 21:33:06 GMT"} })
2623	2024-12-17T21:33:06.587Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
2624	2024-12-17T21:33:06.587Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
2625	2024-12-17T21:33:06.587Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
2626	2024-12-17T21:33:06.588Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 652.693794ms iteration = 21 status = {}
2627	2024-12-17T21:33:06.588Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2628	2024-12-17T21:33:06.591Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
2629	2024-12-17T21:33:06.591Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2630	2024-12-17T21:33:06.592Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2631	2024-12-17T21:33:06.623Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
2632	2024-12-17T21:33:06.623Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "2f13590d-2865-42a0-bf16-40b86b4e47e3", "date": "Tue, 17 Dec 2024 21:33:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2633	2024-12-17T21:33:06.623Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2634	2024-12-17T21:33:06.626Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "b576b352-aec6-4c49-a06e-77318e9202af", "date": "Tue, 17 Dec 2024 21:33:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2635	2024-12-17T21:33:06.626Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2636	2024-12-17T21:33:06.628Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2637	2024-12-17T21:33:06.628Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 692.707857ms iteration = 13 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
2638	2024-12-17T21:33:06.677Z	DEBG	nexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
2639	2024-12-17T21:33:06.677Z	DEBG	nexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
2640	2024-12-17T21:33:06.679Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 743.644716ms iteration = 21 status = null
2641	2024-12-17T21:33:06.679Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2642	2024-12-17T21:33:06.680Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2643	2024-12-17T21:33:06.682Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2644	2024-12-17T21:33:06.682Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2645	2024-12-17T21:33:06.685Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2646	2024-12-17T21:33:06.685Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2647	2024-12-17T21:33:06.688Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2648	2024-12-17T21:33:06.688Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2649	2024-12-17T21:33:06.690Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2650	2024-12-17T21:33:06.690Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2651	2024-12-17T21:33:06.692Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2652	2024-12-17T21:33:06.692Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2653	2024-12-17T21:33:06.692Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
2654	2024-12-17T21:33:06.693Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "ec689df4-c8cb-407c-83c3-083940f33dda", "date": "Tue, 17 Dec 2024 21:33:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
2655	2024-12-17T21:33:06.693Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
2656	2024-12-17T21:33:06.693Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2657	2024-12-17T21:33:06.695Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "7c765d63-7813-466c-a900-04bd7d614792", "date": "Tue, 17 Dec 2024 21:33:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
2658	2024-12-17T21:33:06.695Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
2659	2024-12-17T21:33:06.697Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
2660	2024-12-17T21:33:06.697Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2661	2024-12-17T21:33:06.697Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "61b7f433-e398-459e-aadb-8ac6372db54a", "date": "Tue, 17 Dec 2024 21:33:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
2662	2024-12-17T21:33:06.697Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
2663	2024-12-17T21:33:06.697Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2664	2024-12-17T21:33:06.699Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2665	2024-12-17T21:33:06.699Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2666	2024-12-17T21:33:06.699Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
2667	2024-12-17T21:33:06.699Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 763.76444ms iteration = 13 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
2668	2024-12-17T21:33:06.721Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2669	2024-12-17T21:33:06.721Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2670	2024-12-17T21:33:06.723Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 1a3e1a79-c78b-4fb4-921b-c5e1e32e8b40, rack_id: 8216c8d1-8011-434d-a57a-2b1ce2e89ad8, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(d6c2617a-6b89-4779-bc70-3002fb0a7e5e) } rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0
2671	2024-12-17T21:33:06.723Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2672	2024-12-17T21:33:06.723Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2673	2024-12-17T21:33:06.728Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2674	2024-12-17T21:33:06.728Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2675	2024-12-17T21:33:06.730Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2676	2024-12-17T21:33:06.733Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
2677	2024-12-17T21:33:06.733Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
2678	2024-12-17T21:33:06.733Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2679	2024-12-17T21:33:06.734Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2680	2024-12-17T21:33:06.734Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "2aee88fe-9110-4fa8-96e9-a30365227069", "content-length": "134", "date": "Tue, 17 Dec 2024 21:33:06 GMT"} })
2681	2024-12-17T21:33:06.734Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
2682	2024-12-17T21:33:06.734Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = d6c2617a-6b89-4779-bc70-3002fb0a7e5e
2683	2024-12-17T21:33:06.734Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
2684	2024-12-17T21:33:06.734Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
2685	2024-12-17T21:33:06.734Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2686	2024-12-17T21:33:06.734Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2687	2024-12-17T21:33:06.734Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2688	2024-12-17T21:33:06.734Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
2689	2024-12-17T21:33:06.734Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2690	2024-12-17T21:33:06.734Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
2691	2024-12-17T21:33:06.735Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
2692	2024-12-17T21:33:06.735Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2693	2024-12-17T21:33:06.735Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2694	2024-12-17T21:33:06.737Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
2695	2024-12-17T21:33:06.737Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "1b495ccc-2e35-4e8b-9510-b17f5b81f087", "content-length": "2", "date": "Tue, 17 Dec 2024 21:33:06 GMT"} })
2696	2024-12-17T21:33:06.737Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
2697	2024-12-17T21:33:06.737Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "68b0fe39-3cda-4d22-972d-8147edf1749e", "content-length": "51", "date": "Tue, 17 Dec 2024 21:33:06 GMT"} })
2698	2024-12-17T21:33:06.737Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0
2699	2024-12-17T21:33:06.737Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
2700	2024-12-17T21:33:06.739Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2701	2024-12-17T21:33:06.739Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2702	2024-12-17T21:33:06.741Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2703	2024-12-17T21:33:06.741Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2704	2024-12-17T21:33:06.742Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2705	2024-12-17T21:33:06.743Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2706	2024-12-17T21:33:06.744Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2707	2024-12-17T21:33:06.744Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2708	2024-12-17T21:33:06.747Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2709	2024-12-17T21:33:06.747Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2710	2024-12-17T21:33:06.748Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2711	2024-12-17T21:33:06.749Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2712	2024-12-17T21:33:06.750Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2713	2024-12-17T21:33:06.750Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2714	2024-12-17T21:33:06.751Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2715	2024-12-17T21:33:06.755Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
2716	2024-12-17T21:33:06.755Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2717	2024-12-17T21:33:06.755Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2718	2024-12-17T21:33:06.756Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2719	2024-12-17T21:33:06.756Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2720	2024-12-17T21:33:06.758Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2721	2024-12-17T21:33:06.758Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2722	2024-12-17T21:33:06.759Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2723	2024-12-17T21:33:06.763Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
2724	2024-12-17T21:33:06.763Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2725	2024-12-17T21:33:06.763Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2726	2024-12-17T21:33:06.771Z	DEBG	nexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 90400ecd-43ab-4b42-bc2c-68e9b0a8cb02, kind: Service { id: 5dff2793-3ce8-4251-ba71-1a446966cbf7 }, name: Name("external-dns-5dff2793-3ce8-4251-ba71-1a446966cbf7"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 201, 90])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 907c8800-a1ea-47ff-bdbf-63402c8c3e67, kind: Service { id: a761a47e-08b9-41c4-9bd8-8d77651dfa62 }, name: Name("external-dns-a761a47e-08b9-41c4-9bd8-8d77651dfa62"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 253, 16])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 1950ee03-4e2e-4a6c-a8f2-88cfbaaaf516, kind: Service { id: 2fd5ca9b-ccee-4a78-97bd-4bd196880cfb }, name: Name("nexus-2fd5ca9b-ccee-4a78-97bd-4bd196880cfb"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 213, 8])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 51cf3c6b-ab6e-43a2-a08f-c7929f83346c, kind: Service { id: 0536d936-3647-4bc5-9d28-7041dd1a1d35 }, name: Name("nexus-0536d936-3647-4bc5-9d28-7041dd1a1d35"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f52af8c0-2e1a-4692-9d7e-5972fdf664ce, kind: Service { id: be092aa0-9f0f-4665-8dd8-c2499ee65aff }, name: Name("nexus-be092aa0-9f0f-4665-8dd8-c2499ee65aff"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 196, 114])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
2727	2024-12-17T21:33:06.771Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2728	2024-12-17T21:33:06.771Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2729	2024-12-17T21:33:06.772Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2730	2024-12-17T21:33:06.774Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
2731	2024-12-17T21:33:06.774Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2732	2024-12-17T21:33:06.774Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2733	2024-12-17T21:33:06.776Z	DEBG	nexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
2734	2024-12-17T21:33:06.776Z	DEBG	nexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 90400ecd-43ab-4b42-bc2c-68e9b0a8cb02, kind: Service { id: 5dff2793-3ce8-4251-ba71-1a446966cbf7 }, name: Name("external-dns-5dff2793-3ce8-4251-ba71-1a446966cbf7"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 201, 90])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 907c8800-a1ea-47ff-bdbf-63402c8c3e67, kind: Service { id: a761a47e-08b9-41c4-9bd8-8d77651dfa62 }, name: Name("external-dns-a761a47e-08b9-41c4-9bd8-8d77651dfa62"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 253, 16])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 1950ee03-4e2e-4a6c-a8f2-88cfbaaaf516, kind: Service { id: 2fd5ca9b-ccee-4a78-97bd-4bd196880cfb }, name: Name("nexus-2fd5ca9b-ccee-4a78-97bd-4bd196880cfb"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 213, 8])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 51cf3c6b-ab6e-43a2-a08f-c7929f83346c, kind: Service { id: 0536d936-3647-4bc5-9d28-7041dd1a1d35 }, name: Name("nexus-0536d936-3647-4bc5-9d28-7041dd1a1d35"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f52af8c0-2e1a-4692-9d7e-5972fdf664ce, kind: Service { id: be092aa0-9f0f-4665-8dd8-c2499ee65aff }, name: Name("nexus-be092aa0-9f0f-4665-8dd8-c2499ee65aff"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 196, 114])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
2735	2024-12-17T21:33:06.776Z	DEBG	nexus (service-firewall-rule-progator): resolved 2 rules for sleds background_task = service_firewall_rule_propagation
2736	2024-12-17T21:33:06.848Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "75f7e840-b308-494e-b52d-b238a9483d5c", "date": "Tue, 17 Dec 2024 21:33:06 GMT"} })
2737	2024-12-17T21:33:06.848Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch = Switch0
2738	2024-12-17T21:33:06.848Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
2739	2024-12-17T21:33:06.849Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "866cd1fa-dae4-4e19-a6d6-9c081b560365", "content-length": "682", "date": "Tue, 17 Dec 2024 21:33:06 GMT"} })
2740	2024-12-17T21:33:06.849Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2741	2024-12-17T21:33:06.849Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2742	2024-12-17T21:33:06.852Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2743	2024-12-17T21:33:06.852Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2744	2024-12-17T21:33:06.856Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2745	2024-12-17T21:33:06.856Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2746	2024-12-17T21:33:06.858Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2747	2024-12-17T21:33:06.858Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2748	2024-12-17T21:33:06.866Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 930.600995ms iteration = 21 status = {}
2749	2024-12-17T21:33:06.888Z	DEBG	nexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: d3278cd4-2587-4f0f-89dd-b4d4669a6ffe, time_created: 2024-12-17T21:23:45.515540Z, time_modified: 2024-12-17T21:23:45.515540Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 8216c8d1-8011-434d-a57a-2b1ce2e89ad8, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(51474595840)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)) }]
2750	2024-12-17T21:33:06.888Z	DEBG	nexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
2751	2024-12-17T21:33:06.888Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2752	2024-12-17T21:33:06.888Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2753	2024-12-17T21:33:06.891Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2754	2024-12-17T21:33:06.891Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2755	2024-12-17T21:33:06.893Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2756	2024-12-17T21:33:06.895Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: d3278cd4-2587-4f0f-89dd-b4d4669a6ffe, lookup_type: ById(d3278cd4-2587-4f0f-89dd-b4d4669a6ffe) } result = Ok(())
2757	2024-12-17T21:33:06.919Z	DEBG	nexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e17b988-e354-43f6-92c3-6194750e4d08", "content-length": "30795", "date": "Tue, 17 Dec 2024 21:33:06 GMT"} })
2758	2024-12-17T21:33:06.927Z	DEBG	nexus (service-firewall-rule-progator): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
2759	2024-12-17T21:33:06.927Z	DEBG	nexus (service-firewall-rule-progator): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "9cb2b0b3-ba0f-4e1f-b02b-3d03cc4a231b", "date": "Tue, 17 Dec 2024 21:33:06 GMT"} })
2760	2024-12-17T21:33:06.928Z	INFO	nexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:496
2761	2024-12-17T21:33:06.928Z	DEBG	nexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 992.627613ms
2762	2024-12-17T21:33:06.928Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 992.636974ms iteration = 3 status = {}
2763	2024-12-17T21:33:06.942Z	DEBG	nexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
2764	2024-12-17T21:33:06.942Z	DEBG	nexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
2765	2024-12-17T21:33:06.942Z	DEBG	nexus (BackgroundTasks): finished collection background_task = inventory_collection
2766	2024-12-17T21:33:06.946Z	DEBG	nexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
2767	2024-12-17T21:33:06.946Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
2768	2024-12-17T21:33:06.948Z	DEBG	nexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2769	2024-12-17T21:33:06.949Z	DEBG	nexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
2770	2024-12-17T21:33:06.949Z	DEBG	nexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
2771	2024-12-17T21:33:06.949Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
2772	2024-12-17T21:33:07.141Z	INFO	nexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 8ef70f64-873e-47e4-a0b4-69dc7ea01c4f file = nexus/db-queries/src/db/datastore/inventory.rs:1010
2773	2024-12-17T21:33:07.141Z	DEBG	nexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 8ef70f64-873e-47e4-a0b4-69dc7ea01c4f time_started = 2024-12-17 21:33:06.321602 UTC
2774	2024-12-17T21:33:07.141Z	DEBG	nexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 1.206258575s iteration = 2 status = {"collection_id":"8ef70f64-873e-47e4-a0b4-69dc7ea01c4f","time_done":"2024-12-17 21:33:06.944307 UTC","time_started":"2024-12-17 21:33:06.321602 UTC"}
2775	2024-12-17T21:33:07.141Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 23 reason = Dependency
2776	2024-12-17T21:33:07.141Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2777	2024-12-17T21:33:07.142Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2778	2024-12-17T21:33:07.143Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2779	2024-12-17T21:33:07.145Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2780	2024-12-17T21:33:07.145Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2781	2024-12-17T21:33:07.145Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2782	2024-12-17T21:33:07.148Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2783	2024-12-17T21:33:07.149Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2784	2024-12-17T21:33:07.150Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2785	2024-12-17T21:33:07.152Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2786	2024-12-17T21:33:07.152Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2787	2024-12-17T21:33:07.153Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2788	2024-12-17T21:33:07.198Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 56.694054ms iteration = 23 status = {"physical_disks_added":0}
2789	2024-12-17T21:33:10.395Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 8ad523eb-ceba-435f-a0c0-a502f88ff573 roles = RoleSet { roles: {} } uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?sort_by=id_ascending
2790	2024-12-17T21:33:10.395Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 8ad523eb-ceba-435f-a0c0-a502f88ff573 resource = Database result = Ok(()) uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?sort_by=id_ascending
2791	2024-12-17T21:33:10.397Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2433 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 8ad523eb-ceba-435f-a0c0-a502f88ff573 response_code = 200 uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?sort_by=id_ascending
2792	2024-12-17T21:33:10.397Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 20ffba02-5ccf-49d8-8ff1-86dd2726edb5 roles = RoleSet { roles: {} } uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMTkzZjYyMi00MGY1LTQyOGYtYmVhMy01MmNjNjdlNzhhMzkifX0%3D
2793	2024-12-17T21:33:10.398Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 20ffba02-5ccf-49d8-8ff1-86dd2726edb5 resource = Database result = Ok(()) uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMTkzZjYyMi00MGY1LTQyOGYtYmVhMy01MmNjNjdlNzhhMzkifX0%3D
2794	2024-12-17T21:33:10.400Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2512 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 20ffba02-5ccf-49d8-8ff1-86dd2726edb5 response_code = 200 uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMTkzZjYyMi00MGY1LTQyOGYtYmVhMy01MmNjNjdlNzhhMzkifX0%3D
2795	2024-12-17T21:33:10.406Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 242 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = 63bf0200-8b89-442b-9480-f55b22438e35 response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
2796	2024-12-17T21:33:15.934Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 62 reason = Timeout
2797	2024-12-17T21:33:15.934Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2798	2024-12-17T21:33:15.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2799	2024-12-17T21:33:15.935Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 62 reason = Timeout
2800	2024-12-17T21:33:15.935Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2801	2024-12-17T21:33:15.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2802	2024-12-17T21:33:15.936Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2803	2024-12-17T21:33:15.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2804	2024-12-17T21:33:15.939Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2805	2024-12-17T21:33:15.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2806	2024-12-17T21:33:15.941Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2807	2024-12-17T21:33:15.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2808	2024-12-17T21:33:15.943Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2809	2024-12-17T21:33:15.945Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
2810	2024-12-17T21:33:15.945Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2811	2024-12-17T21:33:15.945Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2812	2024-12-17T21:33:15.956Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
2813	2024-12-17T21:33:15.956Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.927954ms iteration = 62 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
2814	2024-12-17T21:33:15.964Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2815	2024-12-17T21:33:15.964Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2816	2024-12-17T21:33:15.991Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 55.698818ms iteration = 62 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
2817	2024-12-17T21:33:20.406Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 264 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = 25dbf653-79df-4202-b8c5-137472b2723f response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
2818	2024-12-17T21:33:25.394Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 527a34c9-65af-4ae0-b8fd-9a2cb0606a80 roles = RoleSet { roles: {} } uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?sort_by=id_ascending
2819	2024-12-17T21:33:25.395Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 527a34c9-65af-4ae0-b8fd-9a2cb0606a80 resource = Database result = Ok(()) uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?sort_by=id_ascending
2820	2024-12-17T21:33:25.397Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3283 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 527a34c9-65af-4ae0-b8fd-9a2cb0606a80 response_code = 200 uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?sort_by=id_ascending
2821	2024-12-17T21:33:25.398Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = bfcd4006-0962-4812-b19c-f88a9d466dd4 roles = RoleSet { roles: {} } uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMTkzZjYyMi00MGY1LTQyOGYtYmVhMy01MmNjNjdlNzhhMzkifX0%3D
2822	2024-12-17T21:33:25.398Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = bfcd4006-0962-4812-b19c-f88a9d466dd4 resource = Database result = Ok(()) uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMTkzZjYyMi00MGY1LTQyOGYtYmVhMy01MmNjNjdlNzhhMzkifX0%3D
2823	2024-12-17T21:33:25.400Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2357 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = bfcd4006-0962-4812-b19c-f88a9d466dd4 response_code = 200 uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMTkzZjYyMi00MGY1LTQyOGYtYmVhMy01MmNjNjdlNzhhMzkifX0%3D
2824	2024-12-17T21:33:25.934Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 63 reason = Timeout
2825	2024-12-17T21:33:25.935Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2826	2024-12-17T21:33:25.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2827	2024-12-17T21:33:25.935Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 63 reason = Timeout
2828	2024-12-17T21:33:25.935Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2829	2024-12-17T21:33:25.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2830	2024-12-17T21:33:25.937Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2831	2024-12-17T21:33:25.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2832	2024-12-17T21:33:25.939Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2833	2024-12-17T21:33:25.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2834	2024-12-17T21:33:25.940Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2835	2024-12-17T21:33:25.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2836	2024-12-17T21:33:25.942Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2837	2024-12-17T21:33:25.945Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
2838	2024-12-17T21:33:25.945Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2839	2024-12-17T21:33:25.945Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2840	2024-12-17T21:33:25.958Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
2841	2024-12-17T21:33:25.958Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.413374ms iteration = 63 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
2842	2024-12-17T21:33:25.965Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2843	2024-12-17T21:33:25.965Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2844	2024-12-17T21:33:25.992Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 56.90688ms iteration = 63 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
2845	2024-12-17T21:33:30.406Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 265 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = 8ad29b5f-bc67-4173-b4e0-7ed47ae141ee response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
2846	2024-12-17T21:33:35.935Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 64 reason = Timeout
2847	2024-12-17T21:33:35.935Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 22 reason = Timeout
2848	2024-12-17T21:33:35.935Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 22 reason = Timeout
2849	2024-12-17T21:33:35.935Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2850	2024-12-17T21:33:35.935Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
2851	2024-12-17T21:33:35.935Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2852	2024-12-17T21:33:35.935Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 22 reason = Timeout
2853	2024-12-17T21:33:35.935Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
2854	2024-12-17T21:33:35.935Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 22 reason = Timeout
2855	2024-12-17T21:33:35.935Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2856	2024-12-17T21:33:35.935Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 22 reason = Timeout
2857	2024-12-17T21:33:35.935Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 22 reason = Timeout
2858	2024-12-17T21:33:35.935Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2859	2024-12-17T21:33:35.935Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 22 reason = Timeout
2860	2024-12-17T21:33:35.935Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
2861	2024-12-17T21:33:35.935Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 24 reason = Timeout
2862	2024-12-17T21:33:35.935Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2863	2024-12-17T21:33:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2864	2024-12-17T21:33:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
2865	2024-12-17T21:33:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
2866	2024-12-17T21:33:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2867	2024-12-17T21:33:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2868	2024-12-17T21:33:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2869	2024-12-17T21:33:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2870	2024-12-17T21:33:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
2871	2024-12-17T21:33:35.936Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 22 reason = Timeout
2872	2024-12-17T21:33:35.936Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 22 reason = Timeout
2873	2024-12-17T21:33:35.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
2874	2024-12-17T21:33:35.936Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
2875	2024-12-17T21:33:35.936Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 22 reason = Timeout
2876	2024-12-17T21:33:35.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2877	2024-12-17T21:33:35.936Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 22 reason = Timeout
2878	2024-12-17T21:33:35.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
2879	2024-12-17T21:33:35.936Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 22 reason = Timeout
2880	2024-12-17T21:33:35.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
2881	2024-12-17T21:33:35.936Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 22 reason = Timeout
2882	2024-12-17T21:33:35.936Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
2883	2024-12-17T21:33:35.936Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 22 reason = Timeout
2884	2024-12-17T21:33:35.936Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2885	2024-12-17T21:33:35.936Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
2886	2024-12-17T21:33:35.936Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2887	2024-12-17T21:33:35.936Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 64 reason = Timeout
2888	2024-12-17T21:33:35.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2889	2024-12-17T21:33:35.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2890	2024-12-17T21:33:35.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
2891	2024-12-17T21:33:35.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
2892	2024-12-17T21:33:35.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
2893	2024-12-17T21:33:35.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
2894	2024-12-17T21:33:35.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
2895	2024-12-17T21:33:35.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2896	2024-12-17T21:33:35.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2897	2024-12-17T21:33:35.937Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2898	2024-12-17T21:33:35.938Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.210856ms iteration = 22 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
2899	2024-12-17T21:33:35.938Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1874, tv_nsec: 71010407 } })
2900	2024-12-17T21:33:35.938Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
2901	2024-12-17T21:33:35.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2902	2024-12-17T21:33:35.939Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2903	2024-12-17T21:33:35.940Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2904	2024-12-17T21:33:35.941Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2905	2024-12-17T21:33:35.942Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2906	2024-12-17T21:33:35.942Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2907	2024-12-17T21:33:35.943Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1874, tv_nsec: 75915650 } })
2908	2024-12-17T21:33:35.943Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
2909	2024-12-17T21:33:35.944Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
2910	2024-12-17T21:33:35.944Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2911	2024-12-17T21:33:35.944Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2912	2024-12-17T21:33:35.944Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2913	2024-12-17T21:33:35.944Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2914	2024-12-17T21:33:35.944Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2915	2024-12-17T21:33:35.945Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
2916	2024-12-17T21:33:35.945Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2917	2024-12-17T21:33:35.945Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2918	2024-12-17T21:33:35.946Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2919	2024-12-17T21:33:35.947Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
2920	2024-12-17T21:33:35.947Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
2921	2024-12-17T21:33:35.948Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
2922	2024-12-17T21:33:35.950Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2923	2024-12-17T21:33:35.951Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2924	2024-12-17T21:33:35.951Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2925	2024-12-17T21:33:35.952Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2926	2024-12-17T21:33:35.952Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1874, tv_nsec: 85402657 } })
2927	2024-12-17T21:33:35.953Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2928	2024-12-17T21:33:35.953Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
2929	2024-12-17T21:33:35.953Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
2930	2024-12-17T21:33:35.953Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2931	2024-12-17T21:33:35.953Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2932	2024-12-17T21:33:35.955Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2933	2024-12-17T21:33:35.955Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2934	2024-12-17T21:33:35.955Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2935	2024-12-17T21:33:35.959Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
2936	2024-12-17T21:33:35.959Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2937	2024-12-17T21:33:35.959Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2938	2024-12-17T21:33:35.973Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
2939	2024-12-17T21:33:35.973Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 38.222604ms iteration = 64 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
2940	2024-12-17T21:33:35.986Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2941	2024-12-17T21:33:35.986Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2942	2024-12-17T21:33:35.988Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 52.08995ms iteration = 22 status = {"errors":[],"records_set_to_done":[]}
2943	2024-12-17T21:33:35.989Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2944	2024-12-17T21:33:35.989Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
2945	2024-12-17T21:33:35.990Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
2946	2024-12-17T21:33:35.990Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2947	2024-12-17T21:33:35.992Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 56.290664ms iteration = 22 status = {"errors":[],"garbage_collect_requested":[]}
2948	2024-12-17T21:33:35.992Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 57.652411ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
2949	2024-12-17T21:33:35.993Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2950	2024-12-17T21:33:35.994Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2951	2024-12-17T21:33:35.995Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2952	2024-12-17T21:33:35.995Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
2953	2024-12-17T21:33:35.995Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2954	2024-12-17T21:33:35.995Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
2955	2024-12-17T21:33:35.997Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2956	2024-12-17T21:33:35.997Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2957	2024-12-17T21:33:35.998Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2958	2024-12-17T21:33:35.998Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
2959	2024-12-17T21:33:35.998Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2960	2024-12-17T21:33:35.999Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
2961	2024-12-17T21:33:36.000Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
2962	2024-12-17T21:33:36.000Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 63.894469ms iteration = 22 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
2963	2024-12-17T21:33:36.000Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
2964	2024-12-17T21:33:36.001Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
2965	2024-12-17T21:33:36.002Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 66.461979ms iteration = 22 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
2966	2024-12-17T21:33:36.003Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2967	2024-12-17T21:33:36.004Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2968	2024-12-17T21:33:36.004Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
2969	2024-12-17T21:33:36.005Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2970	2024-12-17T21:33:36.005Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2971	2024-12-17T21:33:36.005Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2972	2024-12-17T21:33:36.006Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 70.044746ms iteration = 64 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
2973	2024-12-17T21:33:36.008Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2974	2024-12-17T21:33:36.010Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2975	2024-12-17T21:33:36.010Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2976	2024-12-17T21:33:36.010Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2977	2024-12-17T21:33:36.014Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
2978	2024-12-17T21:33:36.014Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2979	2024-12-17T21:33:36.014Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "65f5d0d5-e205-4fd2-a499-40ffa0efce56", "content-length": "26", "date": "Tue, 17 Dec 2024 21:33:35 GMT"} })
2980	2024-12-17T21:33:36.014Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2981	2024-12-17T21:33:36.016Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
2982	2024-12-17T21:33:36.035Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2983	2024-12-17T21:33:36.038Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2984	2024-12-17T21:33:36.038Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
2985	2024-12-17T21:33:36.038Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 101.851086ms iteration = 22 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
2986	2024-12-17T21:33:36.038Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: d3278cd4-2587-4f0f-89dd-b4d4669a6ffe, lookup_type: ById(d3278cd4-2587-4f0f-89dd-b4d4669a6ffe) } result = Ok(())
2987	2024-12-17T21:33:36.038Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2988	2024-12-17T21:33:36.038Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2989	2024-12-17T21:33:36.040Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 105.438604ms iteration = 22 status = 0
2990	2024-12-17T21:33:36.041Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2991	2024-12-17T21:33:36.041Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2992	2024-12-17T21:33:36.074Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 138.033765ms iteration = 22 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
2993	2024-12-17T21:33:36.100Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2994	2024-12-17T21:33:36.101Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2995	2024-12-17T21:33:36.113Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 178.706147ms iteration = 24 status = {"physical_disks_added":0}
2996	2024-12-17T21:33:36.129Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
2997	2024-12-17T21:33:36.129Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "e1f842eb-fd75-4149-9dec-7be870d3df15", "content-length": "1", "date": "Tue, 17 Dec 2024 21:33:35 GMT"} })
2998	2024-12-17T21:33:36.130Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2999	2024-12-17T21:33:36.131Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3000	2024-12-17T21:33:36.132Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
3001	2024-12-17T21:33:36.132Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3002	2024-12-17T21:33:36.132Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3003	2024-12-17T21:33:36.132Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3004	2024-12-17T21:33:36.133Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 198.606275ms iteration = 22 status = null
3005	2024-12-17T21:33:36.151Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
3006	2024-12-17T21:33:36.151Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3007	2024-12-17T21:33:36.152Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "380245d0-c066-433a-b699-47f1d118f6d4", "content-length": "26", "date": "Tue, 17 Dec 2024 21:33:35 GMT"} })
3008	2024-12-17T21:33:36.152Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3009	2024-12-17T21:33:36.153Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3010	2024-12-17T21:33:36.159Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 zone_address = fd00:1122:3344:101::2
3011	2024-12-17T21:33:36.159Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3012	2024-12-17T21:33:36.160Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "832b3a87-1318-412b-bd6f-94e34435edd6", "content-length": "26", "date": "Tue, 17 Dec 2024 21:33:35 GMT"} })
3013	2024-12-17T21:33:36.160Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3014	2024-12-17T21:33:36.161Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
3015	2024-12-17T21:33:36.162Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3016	2024-12-17T21:33:36.164Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
3017	2024-12-17T21:33:36.164Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3018	2024-12-17T21:33:36.164Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3019	2024-12-17T21:33:36.197Z	INFO	nexus (BackgroundTasks): VPC route manager sled d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
3020	2024-12-17T21:33:36.197Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
3021	2024-12-17T21:33:36.197Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "fec955ca-8bbd-4d7b-b415-49be7ac3fe37", "content-length": "372", "date": "Tue, 17 Dec 2024 21:33:36 GMT"} })
3022	2024-12-17T21:33:36.198Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3023	2024-12-17T21:33:36.198Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3024	2024-12-17T21:33:36.238Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
3025	2024-12-17T21:33:36.238Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
3026	2024-12-17T21:33:36.238Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "ac163c6c-abf9-44d0-8d46-890881603d07", "date": "Tue, 17 Dec 2024 21:33:36 GMT"} })
3027	2024-12-17T21:33:36.238Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3028	2024-12-17T21:33:36.239Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3029	2024-12-17T21:33:36.241Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3030	2024-12-17T21:33:36.241Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3031	2024-12-17T21:33:36.244Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3032	2024-12-17T21:33:36.244Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3033	2024-12-17T21:33:36.248Z	INFO	nexus (BackgroundTasks): VPC route manager sled d3278cd4-2587-4f0f-89dd-b4d4669a6ffe push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
3034	2024-12-17T21:33:36.248Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
3035	2024-12-17T21:33:36.249Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "02eb3679-c00e-4670-9beb-88c4cad4ed07", "date": "Tue, 17 Dec 2024 21:33:36 GMT"} })
3036	2024-12-17T21:33:36.250Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 314.43558ms iteration = 22 status = {}
3037	2024-12-17T21:33:36.341Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 406.078ms iteration = 22 status = {}
3038	2024-12-17T21:33:36.354Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
3039	2024-12-17T21:33:36.355Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec573d5f-d497-403e-b48a-c640d1c248b3", "content-length": "691", "date": "Tue, 17 Dec 2024 21:33:36 GMT"} })
3040	2024-12-17T21:33:36.355Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
3041	2024-12-17T21:33:36.355Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
3042	2024-12-17T21:33:36.355Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
3043	2024-12-17T21:33:36.357Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 422.496235ms iteration = 22 status = {}
3044	2024-12-17T21:33:36.411Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3045	2024-12-17T21:33:36.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3046	2024-12-17T21:33:36.413Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 1a3e1a79-c78b-4fb4-921b-c5e1e32e8b40, rack_id: 8216c8d1-8011-434d-a57a-2b1ce2e89ad8, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(d6c2617a-6b89-4779-bc70-3002fb0a7e5e) } rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0
3047	2024-12-17T21:33:36.413Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3048	2024-12-17T21:33:36.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3049	2024-12-17T21:33:36.427Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
3050	2024-12-17T21:33:36.427Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "e07a1399-989c-41f6-8878-61563729f688", "content-length": "134", "date": "Tue, 17 Dec 2024 21:33:35 GMT"} })
3051	2024-12-17T21:33:36.427Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
3052	2024-12-17T21:33:36.427Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = d6c2617a-6b89-4779-bc70-3002fb0a7e5e
3053	2024-12-17T21:33:36.427Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
3054	2024-12-17T21:33:36.427Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
3055	2024-12-17T21:33:36.427Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3056	2024-12-17T21:33:36.427Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3057	2024-12-17T21:33:36.427Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3058	2024-12-17T21:33:36.427Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
3059	2024-12-17T21:33:36.427Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3060	2024-12-17T21:33:36.427Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
3061	2024-12-17T21:33:36.428Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
3062	2024-12-17T21:33:36.428Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3063	2024-12-17T21:33:36.428Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3064	2024-12-17T21:33:36.430Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
3065	2024-12-17T21:33:36.430Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "b4441829-dead-4e46-8fa9-eedace6a91b2", "content-length": "2", "date": "Tue, 17 Dec 2024 21:33:35 GMT"} })
3066	2024-12-17T21:33:36.430Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
3067	2024-12-17T21:33:36.430Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "0dece47c-6916-48ab-86b9-1fc0b9a41a5f", "content-length": "51", "date": "Tue, 17 Dec 2024 21:33:35 GMT"} })
3068	2024-12-17T21:33:36.430Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0
3069	2024-12-17T21:33:36.430Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
3070	2024-12-17T21:33:36.524Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "8391d3b0-661e-486e-b13b-75d9b981cfd4", "date": "Tue, 17 Dec 2024 21:33:36 GMT"} })
3071	2024-12-17T21:33:36.524Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch = Switch0
3072	2024-12-17T21:33:36.524Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
3073	2024-12-17T21:33:36.527Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "aacd0eeb-2e65-4d05-88a6-13ddd3337b36", "content-length": "682", "date": "Tue, 17 Dec 2024 21:33:36 GMT"} })
3074	2024-12-17T21:33:36.527Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3075	2024-12-17T21:33:36.527Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3076	2024-12-17T21:33:36.531Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3077	2024-12-17T21:33:36.532Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3078	2024-12-17T21:33:36.535Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3079	2024-12-17T21:33:36.535Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3080	2024-12-17T21:33:36.537Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3081	2024-12-17T21:33:36.537Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3082	2024-12-17T21:33:36.544Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 609.27342ms iteration = 22 status = {}
3083	2024-12-17T21:33:40.394Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = a06c665a-43cc-455a-8ef8-6404ecdd949f roles = RoleSet { roles: {} } uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?sort_by=id_ascending
3084	2024-12-17T21:33:40.394Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = a06c665a-43cc-455a-8ef8-6404ecdd949f resource = Database result = Ok(()) uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?sort_by=id_ascending
3085	2024-12-17T21:33:40.397Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2951 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = a06c665a-43cc-455a-8ef8-6404ecdd949f response_code = 200 uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?sort_by=id_ascending
3086	2024-12-17T21:33:40.397Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 5af0f416-4879-47bf-be3d-3ec848e6ada3 roles = RoleSet { roles: {} } uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMTkzZjYyMi00MGY1LTQyOGYtYmVhMy01MmNjNjdlNzhhMzkifX0%3D
3087	2024-12-17T21:33:40.397Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 5af0f416-4879-47bf-be3d-3ec848e6ada3 resource = Database result = Ok(()) uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMTkzZjYyMi00MGY1LTQyOGYtYmVhMy01MmNjNjdlNzhhMzkifX0%3D
3088	2024-12-17T21:33:40.400Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2588 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 5af0f416-4879-47bf-be3d-3ec848e6ada3 response_code = 200 uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMTkzZjYyMi00MGY1LTQyOGYtYmVhMy01MmNjNjdlNzhhMzkifX0%3D
3089	2024-12-17T21:33:40.406Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 238 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = 1a6729fa-77e6-4753-b7cd-fe95e60401d2 response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
3090	2024-12-17T21:33:45.934Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 65 reason = Timeout
3091	2024-12-17T21:33:45.934Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3092	2024-12-17T21:33:45.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3093	2024-12-17T21:33:45.935Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 65 reason = Timeout
3094	2024-12-17T21:33:45.935Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3095	2024-12-17T21:33:45.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3096	2024-12-17T21:33:45.936Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3097	2024-12-17T21:33:45.937Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3098	2024-12-17T21:33:45.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3099	2024-12-17T21:33:45.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3100	2024-12-17T21:33:45.939Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3101	2024-12-17T21:33:45.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3102	2024-12-17T21:33:45.941Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3103	2024-12-17T21:33:45.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3104	2024-12-17T21:33:45.942Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3105	2024-12-17T21:33:45.945Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
3106	2024-12-17T21:33:45.945Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3107	2024-12-17T21:33:45.945Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3108	2024-12-17T21:33:45.953Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
3109	2024-12-17T21:33:45.954Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.404265ms iteration = 65 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
3110	2024-12-17T21:33:45.965Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 30.524135ms iteration = 65 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
3111	2024-12-17T21:33:50.406Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 321 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = b157eac7-9285-4638-a409-f930790ce194 response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
3112	2024-12-17T21:33:55.934Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 66 reason = Timeout
3113	2024-12-17T21:33:55.934Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3114	2024-12-17T21:33:55.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3115	2024-12-17T21:33:55.935Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 66 reason = Timeout
3116	2024-12-17T21:33:55.935Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3117	2024-12-17T21:33:55.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3118	2024-12-17T21:33:55.936Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3119	2024-12-17T21:33:55.937Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3120	2024-12-17T21:33:55.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3121	2024-12-17T21:33:55.938Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3122	2024-12-17T21:33:55.938Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3123	2024-12-17T21:33:55.938Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.239018ms iteration = 66 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
3124	2024-12-17T21:33:55.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3125	2024-12-17T21:33:55.940Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3126	2024-12-17T21:33:55.940Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3127	2024-12-17T21:33:55.941Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3128	2024-12-17T21:33:55.944Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
3129	2024-12-17T21:33:55.944Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3130	2024-12-17T21:33:55.944Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3131	2024-12-17T21:33:55.951Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
3132	2024-12-17T21:33:55.951Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.00557ms iteration = 66 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
3133	2024-12-17T21:34:00.406Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 323 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = ef859918-c729-4793-9f6c-1fe7190bf783 response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
3134	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 12 reason = Timeout
3135	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 13 reason = Timeout
3136	2024-12-17T21:34:05.934Z	WARN	nexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5
3137	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 13.83\u{b5}s iteration = 13 status = {"enabled":false,"target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5"}
3138	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 23 reason = Timeout
3139	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = external_endpoints iteration = 12 reason = Timeout
3140	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3141	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 23 reason = Timeout
3142	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3143	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 23 reason = Timeout
3144	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 12 reason = Timeout
3145	2024-12-17T21:34:05.934Z	INFO	nexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-17 21:24:05.934493691 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
3146	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
3147	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 12 reason = Timeout
3148	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 67 reason = Timeout
3149	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
3150	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
3151	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3152	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 12 reason = Timeout
3153	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
3154	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3155	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3156	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 14 reason = Timeout
3157	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 12 reason = Timeout
3158	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 23 reason = Timeout
3159	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3160	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 14 reason = Timeout
3161	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 25 reason = Timeout
3162	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3163	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 23 reason = Timeout
3164	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3165	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 12 reason = Timeout
3166	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 23 reason = Timeout
3167	2024-12-17T21:34:05.934Z	INFO	nexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
3168	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
3169	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
3170	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 23 reason = Timeout
3171	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3172	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3173	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3174	2024-12-17T21:34:05.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
3175	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3176	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3177	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3178	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
3179	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3180	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3181	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3182	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
3183	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
3184	2024-12-17T21:34:05.935Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0005c310-624c-4312-b0fd-35f571746851.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6ac4d5e8-fcd9-4aa9-aed5-fb3dc293aac2.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9b804364-993d-47ba-81a1-d0ef32f96529.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("9b804364-993d-47ba-81a1-d0ef32f96529.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1904, tv_nsec: 67908293 } })
3185	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3186	2024-12-17T21:34:05.935Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("5dff2793-3ce8-4251-ba71-1a446966cbf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a761a47e-08b9-41c4-9bd8-8d77651dfa62.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a761a47e-08b9-41c4-9bd8-8d77651dfa62.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1904, tv_nsec: 67922384 } })
3187	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 23 reason = Timeout
3188	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 23 reason = Timeout
3189	2024-12-17T21:34:05.935Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
3190	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
3191	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3192	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 12 reason = Timeout
3193	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
3194	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 23 reason = Timeout
3195	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
3196	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 67 reason = Timeout
3197	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3198	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 12 reason = Timeout
3199	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
3200	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 23 reason = Timeout
3201	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
3202	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 23 reason = Timeout
3203	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
3204	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 23 reason = Timeout
3205	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
3206	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
3207	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 12 reason = Timeout
3208	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 23 reason = Timeout
3209	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
3210	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
3211	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
3212	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3213	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3214	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
3215	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
3216	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
3217	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
3218	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
3219	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
3220	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.322376ms iteration = 12 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
3221	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
3222	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
3223	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
3224	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.502011ms iteration = 12 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
3225	2024-12-17T21:34:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
3226	2024-12-17T21:34:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3227	2024-12-17T21:34:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3228	2024-12-17T21:34:05.938Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3229	2024-12-17T21:34:05.938Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.41594ms iteration = 23 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
3230	2024-12-17T21:34:05.939Z	INFO	nexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-17 21:24:05.934493691 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
3231	2024-12-17T21:34:05.939Z	DEBG	nexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.529713ms iteration = 12 status = {"expiration":"2024-12-17T21:24:05.934493691Z","pruned":[]}
3232	2024-12-17T21:34:05.939Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3233	2024-12-17T21:34:05.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3234	2024-12-17T21:34:05.940Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
3235	2024-12-17T21:34:05.940Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3236	2024-12-17T21:34:05.940Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3237	2024-12-17T21:34:05.940Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1904, tv_nsec: 72924829 } })
3238	2024-12-17T21:34:05.940Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
3239	2024-12-17T21:34:05.941Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3240	2024-12-17T21:34:05.942Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3241	2024-12-17T21:34:05.942Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3242	2024-12-17T21:34:05.944Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
3243	2024-12-17T21:34:05.944Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
3244	2024-12-17T21:34:05.944Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
3245	2024-12-17T21:34:05.945Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1904, tv_nsec: 77912834 } })
3246	2024-12-17T21:34:05.945Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
3247	2024-12-17T21:34:05.945Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3248	2024-12-17T21:34:05.945Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
3249	2024-12-17T21:34:05.945Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3250	2024-12-17T21:34:05.945Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3251	2024-12-17T21:34:05.946Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3252	2024-12-17T21:34:05.946Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3253	2024-12-17T21:34:05.947Z	DEBG	nexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 12.503188ms iteration = 12 status = {"errors":[],"found_port_ok":[]}
3254	2024-12-17T21:34:05.947Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3255	2024-12-17T21:34:05.950Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
3256	2024-12-17T21:34:05.950Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3257	2024-12-17T21:34:05.950Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3258	2024-12-17T21:34:05.951Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3259	2024-12-17T21:34:05.954Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
3260	2024-12-17T21:34:05.954Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
3261	2024-12-17T21:34:05.954Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
3262	2024-12-17T21:34:05.958Z	DEBG	nexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
3263	2024-12-17T21:34:05.958Z	INFO	nexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
3264	2024-12-17T21:34:05.958Z	DEBG	nexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 23.124156ms iteration = 12 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
3265	2024-12-17T21:34:05.965Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3266	2024-12-17T21:34:05.965Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3267	2024-12-17T21:34:05.967Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3268	2024-12-17T21:34:05.967Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3269	2024-12-17T21:34:05.967Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3270	2024-12-17T21:34:05.967Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3271	2024-12-17T21:34:05.969Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3272	2024-12-17T21:34:05.973Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
3273	2024-12-17T21:34:05.973Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3274	2024-12-17T21:34:05.973Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3275	2024-12-17T21:34:05.975Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3276	2024-12-17T21:34:05.978Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
3277	2024-12-17T21:34:05.978Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3278	2024-12-17T21:34:05.978Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3279	2024-12-17T21:34:05.979Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3280	2024-12-17T21:34:05.979Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3281	2024-12-17T21:34:05.979Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3282	2024-12-17T21:34:05.982Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3283	2024-12-17T21:34:05.982Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
3284	2024-12-17T21:34:05.982Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3285	2024-12-17T21:34:05.982Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3286	2024-12-17T21:34:05.983Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3287	2024-12-17T21:34:05.983Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
3288	2024-12-17T21:34:05.984Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
3289	2024-12-17T21:34:05.984Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 49.035077ms iteration = 67 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
3290	2024-12-17T21:34:05.987Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
3291	2024-12-17T21:34:05.988Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
3292	2024-12-17T21:34:05.996Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3293	2024-12-17T21:34:05.998Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
3294	2024-12-17T21:34:05.998Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
3295	2024-12-17T21:34:05.999Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
3296	2024-12-17T21:34:05.999Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 65.05035ms iteration = 67 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
3297	2024-12-17T21:34:06.000Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
3298	2024-12-17T21:34:06.000Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3299	2024-12-17T21:34:06.000Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3300	2024-12-17T21:34:06.004Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 67.948639ms iteration = 23 status = {"errors":[],"garbage_collect_requested":[]}
3301	2024-12-17T21:34:06.008Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3302	2024-12-17T21:34:06.008Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3303	2024-12-17T21:34:06.008Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
3304	2024-12-17T21:34:06.008Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 73.072868ms iteration = 23 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
3305	2024-12-17T21:34:06.009Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3306	2024-12-17T21:34:06.009Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3307	2024-12-17T21:34:06.011Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3308	2024-12-17T21:34:06.011Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
3309	2024-12-17T21:34:06.011Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3310	2024-12-17T21:34:06.011Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3311	2024-12-17T21:34:06.012Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 77.518318ms iteration = 23 status = {"errors":[],"records_set_to_done":[]}
3312	2024-12-17T21:34:06.013Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
3313	2024-12-17T21:34:06.014Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
3314	2024-12-17T21:34:06.014Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
3315	2024-12-17T21:34:06.014Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
3316	2024-12-17T21:34:06.014Z	DEBG	nexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
3317	2024-12-17T21:34:06.014Z	INFO	nexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
3318	2024-12-17T21:34:06.014Z	DEBG	nexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 80.192011ms iteration = 12 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
3319	2024-12-17T21:34:06.014Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
3320	2024-12-17T21:34:06.015Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
3321	2024-12-17T21:34:06.015Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1904, tv_nsec: 147986641 } })
3322	2024-12-17T21:34:06.015Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
3323	2024-12-17T21:34:06.018Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
3324	2024-12-17T21:34:06.018Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
3325	2024-12-17T21:34:06.018Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
3326	2024-12-17T21:34:06.018Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
3327	2024-12-17T21:34:06.019Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3328	2024-12-17T21:34:06.020Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
3329	2024-12-17T21:34:06.021Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3330	2024-12-17T21:34:06.021Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3331	2024-12-17T21:34:06.032Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3332	2024-12-17T21:34:06.034Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
3333	2024-12-17T21:34:06.034Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3334	2024-12-17T21:34:06.034Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 99.823842ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
3335	2024-12-17T21:34:06.034Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3336	2024-12-17T21:34:06.034Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
3337	2024-12-17T21:34:06.035Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
3338	2024-12-17T21:34:06.044Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 110.016387ms iteration = 25 status = {"physical_disks_added":0}
3339	2024-12-17T21:34:06.045Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3340	2024-12-17T21:34:06.046Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3341	2024-12-17T21:34:06.048Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3342	2024-12-17T21:34:06.048Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3343	2024-12-17T21:34:06.050Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 115.358442ms iteration = 23 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
3344	2024-12-17T21:34:06.050Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3345	2024-12-17T21:34:06.052Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
3346	2024-12-17T21:34:06.052Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
3347	2024-12-17T21:34:06.052Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 116.933735ms iteration = 23 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
3348	2024-12-17T21:34:06.054Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: d3278cd4-2587-4f0f-89dd-b4d4669a6ffe, lookup_type: ById(d3278cd4-2587-4f0f-89dd-b4d4669a6ffe) } result = Ok(())
3349	2024-12-17T21:34:06.054Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3350	2024-12-17T21:34:06.055Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3351	2024-12-17T21:34:06.055Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
3352	2024-12-17T21:34:06.055Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
3353	2024-12-17T21:34:06.055Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
3354	2024-12-17T21:34:06.055Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
3355	2024-12-17T21:34:06.055Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
3356	2024-12-17T21:34:06.055Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3357	2024-12-17T21:34:06.055Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3358	2024-12-17T21:34:06.056Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 122.31661ms iteration = 23 status = 0
3359	2024-12-17T21:34:06.058Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
3360	2024-12-17T21:34:06.058Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
3361	2024-12-17T21:34:06.058Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
3362	2024-12-17T21:34:06.058Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3363	2024-12-17T21:34:06.058Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3364	2024-12-17T21:34:06.060Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3365	2024-12-17T21:34:06.062Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
3366	2024-12-17T21:34:06.062Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3367	2024-12-17T21:34:06.063Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3368	2024-12-17T21:34:06.069Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
3369	2024-12-17T21:34:06.069Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 574f0c35-b8f2-48fe-831c-78afc4a62aaa dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
3370	2024-12-17T21:34:06.069Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3371	2024-12-17T21:34:06.069Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3372	2024-12-17T21:34:06.071Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3373	2024-12-17T21:34:06.072Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
3374	2024-12-17T21:34:06.072Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3375	2024-12-17T21:34:06.073Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "aee425f9-6979-4f26-910e-6d0b9e8bbed4", "content-length": "26", "date": "Tue, 17 Dec 2024 21:34:05 GMT"} })
3376	2024-12-17T21:34:06.073Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3377	2024-12-17T21:34:06.074Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3378	2024-12-17T21:34:06.074Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
3379	2024-12-17T21:34:06.074Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3380	2024-12-17T21:34:06.075Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3381	2024-12-17T21:34:06.077Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 574f0c35-b8f2-48fe-831c-78afc4a62aaa dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
3382	2024-12-17T21:34:06.077Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
3383	2024-12-17T21:34:06.077Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-17 21:23:48.446255 UTC dns_group = internal generation = 1
3384	2024-12-17T21:34:06.077Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 143.318509ms iteration = 12 status = {"generation":1}
3385	2024-12-17T21:34:06.084Z	DEBG	nexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-17T21:23:03.821125Z, time_modified: 2024-12-17T21:23:03.821125Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 81e78e1e-e55a-44f9-abdc-c9c916e1d9fd, db_silo: Silo { identity: SiloIdentity { id: 81e78e1e-e55a-44f9-abdc-c9c916e1d9fd, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-17T21:23:49.443930Z, time_modified: 2024-12-17T21:23:49.443930Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e275643b21ab02733b9616c9cd3b0193e27c7f6294b7d310d6f1726c72768e50" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 81e78e1e-e55a-44f9-abdc-c9c916e1d9fd, db_silo: Silo { identity: SiloIdentity { id: 81e78e1e-e55a-44f9-abdc-c9c916e1d9fd, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-17T21:23:49.443930Z, time_modified: 2024-12-17T21:23:49.443930Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e275643b21ab02733b9616c9cd3b0193e27c7f6294b7d310d6f1726c72768e50" }] }) }
3386	2024-12-17T21:34:06.084Z	DEBG	nexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 149.65399ms iteration = 12 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"81e78e1e-e55a-44f9-abdc-c9c916e1d9fd","tls_certs":["e275643b21ab02733b9616c9cd3b0193e27c7f6294b7d310d6f1726c72768e50"]}},"default_endpoint":{"silo_id":"81e78e1e-e55a-44f9-abdc-c9c916e1d9fd","tls_certs":["e275643b21ab02733b9616c9cd3b0193e27c7f6294b7d310d6f1726c72768e50"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
3387	2024-12-17T21:34:06.085Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
3388	2024-12-17T21:34:06.085Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3389	2024-12-17T21:34:06.085Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "fef74515-e6db-4f36-81f2-aaf73583d9ca", "content-length": "26", "date": "Tue, 17 Dec 2024 21:34:05 GMT"} })
3390	2024-12-17T21:34:06.085Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3391	2024-12-17T21:34:06.087Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3392	2024-12-17T21:34:06.092Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3393	2024-12-17T21:34:06.093Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3394	2024-12-17T21:34:06.104Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3395	2024-12-17T21:34:06.104Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3396	2024-12-17T21:34:06.106Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 171.089121ms iteration = 23 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
3397	2024-12-17T21:34:06.106Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3398	2024-12-17T21:34:06.106Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3399	2024-12-17T21:34:06.109Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
3400	2024-12-17T21:34:06.109Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3401	2024-12-17T21:34:06.109Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3402	2024-12-17T21:34:06.109Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
3403	2024-12-17T21:34:06.109Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3404	2024-12-17T21:34:06.109Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3405	2024-12-17T21:34:06.110Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3406	2024-12-17T21:34:06.112Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
3407	2024-12-17T21:34:06.112Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3408	2024-12-17T21:34:06.112Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3409	2024-12-17T21:34:06.133Z	WARN	nexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
3410	2024-12-17T21:34:06.133Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 197.581877ms iteration = 12 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
3411	2024-12-17T21:34:06.145Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
3412	2024-12-17T21:34:06.145Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = ea6acdde-77d4-4bf6-b578-d9a0a29ac37b dns_zone_name = oxide.test found_so_far = 0 version = 2
3413	2024-12-17T21:34:06.145Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3414	2024-12-17T21:34:06.145Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3415	2024-12-17T21:34:06.147Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3416	2024-12-17T21:34:06.149Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
3417	2024-12-17T21:34:06.149Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3418	2024-12-17T21:34:06.150Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3419	2024-12-17T21:34:06.151Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = ea6acdde-77d4-4bf6-b578-d9a0a29ac37b dns_zone_name = oxide.test found_so_far = 1 version = 2
3420	2024-12-17T21:34:06.151Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
3421	2024-12-17T21:34:06.152Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-17 21:23:49.491999 UTC dns_group = external generation = 2
3422	2024-12-17T21:34:06.152Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 217.463915ms iteration = 12 status = {"generation":2}
3423	2024-12-17T21:34:06.208Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
3424	2024-12-17T21:34:06.217Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
3425	2024-12-17T21:34:06.238Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 zone_address = fd00:1122:3344:101::2
3426	2024-12-17T21:34:06.238Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3427	2024-12-17T21:34:06.238Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8add7e56-0b9e-49d1-afda-68abec0d224f", "content-length": "26", "date": "Tue, 17 Dec 2024 21:34:05 GMT"} })
3428	2024-12-17T21:34:06.238Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3429	2024-12-17T21:34:06.240Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
3430	2024-12-17T21:34:06.308Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 373.718284ms iteration = 23 status = {}
3431	2024-12-17T21:34:06.323Z	INFO	nexus (BackgroundTasks): VPC route manager sled d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
3432	2024-12-17T21:34:06.323Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
3433	2024-12-17T21:34:06.323Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "5aa73c42-743c-49f4-9f93-bbaab3b3c3bc", "content-length": "372", "date": "Tue, 17 Dec 2024 21:34:06 GMT"} })
3434	2024-12-17T21:34:06.323Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3435	2024-12-17T21:34:06.323Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3436	2024-12-17T21:34:06.335Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
3437	2024-12-17T21:34:06.335Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "209c2bcd-49ed-4589-ae84-cb6ca6fe7091", "content-length": "1", "date": "Tue, 17 Dec 2024 21:34:06 GMT"} })
3438	2024-12-17T21:34:06.337Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3439	2024-12-17T21:34:06.337Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3440	2024-12-17T21:34:06.338Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 404.284132ms iteration = 23 status = null
3441	2024-12-17T21:34:06.365Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
3442	2024-12-17T21:34:06.365Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
3443	2024-12-17T21:34:06.365Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "f95b3c72-9bee-484b-8a43-78a20bf15402", "date": "Tue, 17 Dec 2024 21:34:06 GMT"} })
3444	2024-12-17T21:34:06.365Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3445	2024-12-17T21:34:06.365Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3446	2024-12-17T21:34:06.393Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
3447	2024-12-17T21:34:06.395Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3448	2024-12-17T21:34:06.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3449	2024-12-17T21:34:06.399Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "1cfd7e7a-0210-4881-b089-238f864ab462", "content-length": "691", "date": "Tue, 17 Dec 2024 21:34:06 GMT"} })
3450	2024-12-17T21:34:06.399Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
3451	2024-12-17T21:34:06.399Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
3452	2024-12-17T21:34:06.399Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
3453	2024-12-17T21:34:06.399Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3454	2024-12-17T21:34:06.399Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3455	2024-12-17T21:34:06.400Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 465.741895ms iteration = 23 status = {}
3456	2024-12-17T21:34:06.403Z	INFO	nexus (BackgroundTasks): VPC route manager sled d3278cd4-2587-4f0f-89dd-b4d4669a6ffe push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
3457	2024-12-17T21:34:06.403Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
3458	2024-12-17T21:34:06.404Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "11c78b01-e41c-4954-8190-1b8fe25b8834", "date": "Tue, 17 Dec 2024 21:34:06 GMT"} })
3459	2024-12-17T21:34:06.405Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 470.197915ms iteration = 23 status = {}
3460	2024-12-17T21:34:06.465Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
3461	2024-12-17T21:34:06.498Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
3462	2024-12-17T21:34:06.498Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "48d163bc-56fd-4165-8083-3c9f974a05ba", "date": "Tue, 17 Dec 2024 21:34:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
3463	2024-12-17T21:34:06.498Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
3464	2024-12-17T21:34:06.500Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "8c2f3a00-91b1-41f3-ba80-b66c2d28de44", "date": "Tue, 17 Dec 2024 21:34:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
3465	2024-12-17T21:34:06.500Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
3466	2024-12-17T21:34:06.502Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
3467	2024-12-17T21:34:06.502Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 567.875208ms iteration = 14 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
3468	2024-12-17T21:34:06.528Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
3469	2024-12-17T21:34:06.529Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3470	2024-12-17T21:34:06.529Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "256af29b-41d5-4734-a309-2556deddca70", "date": "Tue, 17 Dec 2024 21:34:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
3471	2024-12-17T21:34:06.529Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
3472	2024-12-17T21:34:06.529Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3473	2024-12-17T21:34:06.531Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "4ccb595f-895f-4748-b444-8752da36d490", "date": "Tue, 17 Dec 2024 21:34:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
3474	2024-12-17T21:34:06.531Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
3475	2024-12-17T21:34:06.531Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 1a3e1a79-c78b-4fb4-921b-c5e1e32e8b40, rack_id: 8216c8d1-8011-434d-a57a-2b1ce2e89ad8, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(d6c2617a-6b89-4779-bc70-3002fb0a7e5e) } rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0
3476	2024-12-17T21:34:06.531Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3477	2024-12-17T21:34:06.531Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3478	2024-12-17T21:34:06.532Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "54b3ff4c-1bd3-4794-8894-4a5467ff1573", "date": "Tue, 17 Dec 2024 21:34:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
3479	2024-12-17T21:34:06.532Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
3480	2024-12-17T21:34:06.533Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
3481	2024-12-17T21:34:06.533Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 599.001951ms iteration = 14 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
3482	2024-12-17T21:34:06.541Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
3483	2024-12-17T21:34:06.541Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "047c522c-4f76-4481-ae4e-ef3c3bc7cfb2", "content-length": "134", "date": "Tue, 17 Dec 2024 21:34:06 GMT"} })
3484	2024-12-17T21:34:06.541Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
3485	2024-12-17T21:34:06.541Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = d6c2617a-6b89-4779-bc70-3002fb0a7e5e
3486	2024-12-17T21:34:06.541Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
3487	2024-12-17T21:34:06.541Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
3488	2024-12-17T21:34:06.541Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3489	2024-12-17T21:34:06.541Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3490	2024-12-17T21:34:06.541Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3491	2024-12-17T21:34:06.541Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
3492	2024-12-17T21:34:06.542Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3493	2024-12-17T21:34:06.542Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
3494	2024-12-17T21:34:06.542Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
3495	2024-12-17T21:34:06.542Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3496	2024-12-17T21:34:06.542Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3497	2024-12-17T21:34:06.544Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
3498	2024-12-17T21:34:06.544Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "782c9bd3-159a-44f2-a8e4-b78f93693236", "content-length": "2", "date": "Tue, 17 Dec 2024 21:34:06 GMT"} })
3499	2024-12-17T21:34:06.544Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
3500	2024-12-17T21:34:06.544Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "a85cd405-3001-41ea-97f7-89cbc2df8aa0", "content-length": "51", "date": "Tue, 17 Dec 2024 21:34:06 GMT"} })
3501	2024-12-17T21:34:06.544Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0
3502	2024-12-17T21:34:06.544Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
3503	2024-12-17T21:34:06.619Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "cf49fe80-86a3-4820-87fd-5fcf96a997ec", "date": "Tue, 17 Dec 2024 21:34:06 GMT"} })
3504	2024-12-17T21:34:06.619Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch = Switch0
3505	2024-12-17T21:34:06.619Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
3506	2024-12-17T21:34:06.620Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "57f77008-ae57-4242-880f-1c4f5f935b60", "content-length": "682", "date": "Tue, 17 Dec 2024 21:34:06 GMT"} })
3507	2024-12-17T21:34:06.620Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3508	2024-12-17T21:34:06.620Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3509	2024-12-17T21:34:06.623Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3510	2024-12-17T21:34:06.623Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3511	2024-12-17T21:34:06.626Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3512	2024-12-17T21:34:06.627Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3513	2024-12-17T21:34:06.629Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3514	2024-12-17T21:34:06.629Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3515	2024-12-17T21:34:06.633Z	DEBG	nexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
3516	2024-12-17T21:34:06.633Z	DEBG	nexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
3517	2024-12-17T21:34:06.635Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 700.789586ms iteration = 23 status = {}
3518	2024-12-17T21:34:06.663Z	DEBG	nexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
3519	2024-12-17T21:34:06.663Z	INFO	nexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:38487
3520	2024-12-17T21:34:06.663Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:38487 req_id = 2de55ebf-e868-4be1-817c-e137ae9a543b roles = RoleSet { roles: {} } uri = /metrics/producers
3521	2024-12-17T21:34:06.664Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:38487 req_id = 2de55ebf-e868-4be1-817c-e137ae9a543b resource = Database result = Ok(()) uri = /metrics/producers
3522	2024-12-17T21:34:06.701Z	INFO	nexus: assigned collector to new producer collector_id = 952238fd-7df7-4f2b-b8ee-8be2d7488464 file = nexus/src/app/oximeter.rs:76 producer_id = 2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
3523	2024-12-17T21:34:06.701Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 37483 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:38487 req_id = 2de55ebf-e868-4be1-817c-e137ae9a543b response_code = 201 uri = /metrics/producers
3524	2024-12-17T21:34:06.701Z	DEBG	nexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "2de55ebf-e868-4be1-817c-e137ae9a543b", "content-length": "41", "date": "Tue, 17 Dec 2024 21:34:06 GMT"} })
3525	2024-12-17T21:34:06.702Z	DEBG	nexus (producer-registration-task): registered with nexus successfully
3526	2024-12-17T21:34:06.702Z	DEBG	nexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
3527	2024-12-17T21:34:10.394Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = d0dd3c63-e871-4770-9c9e-a0dfa3ce3c1c roles = RoleSet { roles: {} } uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?sort_by=id_ascending
3528	2024-12-17T21:34:10.394Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = d0dd3c63-e871-4770-9c9e-a0dfa3ce3c1c resource = Database result = Ok(()) uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?sort_by=id_ascending
3529	2024-12-17T21:34:10.397Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3037 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = d0dd3c63-e871-4770-9c9e-a0dfa3ce3c1c response_code = 200 uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?sort_by=id_ascending
3530	2024-12-17T21:34:10.397Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 4469ceef-f869-4950-a84b-a7d27222fe4b roles = RoleSet { roles: {} } uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMTkzZjYyMi00MGY1LTQyOGYtYmVhMy01MmNjNjdlNzhhMzkifX0%3D
3531	2024-12-17T21:34:10.398Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 4469ceef-f869-4950-a84b-a7d27222fe4b resource = Database result = Ok(()) uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMTkzZjYyMi00MGY1LTQyOGYtYmVhMy01MmNjNjdlNzhhMzkifX0%3D
3532	2024-12-17T21:34:10.400Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2543 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 4469ceef-f869-4950-a84b-a7d27222fe4b response_code = 200 uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMTkzZjYyMi00MGY1LTQyOGYtYmVhMy01MmNjNjdlNzhhMzkifX0%3D
3533	2024-12-17T21:34:10.406Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 252 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = 187dba7d-9960-408f-9a85-f3591e2ba012 response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
3534	2024-12-17T21:34:15.934Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 68 reason = Timeout
3535	2024-12-17T21:34:15.934Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3536	2024-12-17T21:34:15.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3537	2024-12-17T21:34:15.935Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 68 reason = Timeout
3538	2024-12-17T21:34:15.935Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3539	2024-12-17T21:34:15.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3540	2024-12-17T21:34:15.937Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3541	2024-12-17T21:34:15.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3542	2024-12-17T21:34:15.939Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3543	2024-12-17T21:34:15.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3544	2024-12-17T21:34:15.942Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3545	2024-12-17T21:34:15.942Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3546	2024-12-17T21:34:15.944Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3547	2024-12-17T21:34:15.946Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
3548	2024-12-17T21:34:15.946Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3549	2024-12-17T21:34:15.947Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3550	2024-12-17T21:34:15.956Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
3551	2024-12-17T21:34:15.956Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.848791ms iteration = 68 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
3552	2024-12-17T21:34:15.971Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3553	2024-12-17T21:34:15.971Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3554	2024-12-17T21:34:15.973Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 37.892925ms iteration = 68 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
3555	2024-12-17T21:34:20.406Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 271 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = 3e9f458c-aeef-4bdf-a10b-5844d3dc2d0d response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
3556	2024-12-17T21:34:25.934Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 69 reason = Timeout
3557	2024-12-17T21:34:25.934Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3558	2024-12-17T21:34:25.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3559	2024-12-17T21:34:25.935Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 69 reason = Timeout
3560	2024-12-17T21:34:25.935Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3561	2024-12-17T21:34:25.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3562	2024-12-17T21:34:25.937Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3563	2024-12-17T21:34:25.938Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3564	2024-12-17T21:34:25.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3565	2024-12-17T21:34:25.940Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3566	2024-12-17T21:34:25.940Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3567	2024-12-17T21:34:25.940Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3568	2024-12-17T21:34:25.941Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 5.221451ms iteration = 69 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
3569	2024-12-17T21:34:25.943Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3570	2024-12-17T21:34:25.944Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3571	2024-12-17T21:34:25.947Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3572	2024-12-17T21:34:25.950Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
3573	2024-12-17T21:34:25.950Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3574	2024-12-17T21:34:25.950Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3575	2024-12-17T21:34:25.966Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
3576	2024-12-17T21:34:25.966Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 31.811041ms iteration = 69 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
3577	2024-12-17T21:34:30.406Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 276 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = 2d24c33a-11bc-48bf-95ca-741dd8894061 response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
3578	2024-12-17T21:34:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 70 reason = Timeout
3579	2024-12-17T21:34:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 24 reason = Timeout
3580	2024-12-17T21:34:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 24 reason = Timeout
3581	2024-12-17T21:34:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3582	2024-12-17T21:34:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3583	2024-12-17T21:34:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 24 reason = Timeout
3584	2024-12-17T21:34:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3585	2024-12-17T21:34:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 24 reason = Timeout
3586	2024-12-17T21:34:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 24 reason = Timeout
3587	2024-12-17T21:34:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 24 reason = Timeout
3588	2024-12-17T21:34:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
3589	2024-12-17T21:34:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
3590	2024-12-17T21:34:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3591	2024-12-17T21:34:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 24 reason = Timeout
3592	2024-12-17T21:34:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3593	2024-12-17T21:34:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 26 reason = Timeout
3594	2024-12-17T21:34:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3595	2024-12-17T21:34:35.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3596	2024-12-17T21:34:35.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3597	2024-12-17T21:34:35.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3598	2024-12-17T21:34:35.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3599	2024-12-17T21:34:35.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3600	2024-12-17T21:34:35.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3601	2024-12-17T21:34:35.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
3602	2024-12-17T21:34:35.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
3603	2024-12-17T21:34:35.935Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 24 reason = Timeout
3604	2024-12-17T21:34:35.935Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 24 reason = Timeout
3605	2024-12-17T21:34:35.935Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
3606	2024-12-17T21:34:35.935Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
3607	2024-12-17T21:34:35.935Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 24 reason = Timeout
3608	2024-12-17T21:34:35.935Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
3609	2024-12-17T21:34:35.935Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 24 reason = Timeout
3610	2024-12-17T21:34:35.935Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
3611	2024-12-17T21:34:35.935Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3612	2024-12-17T21:34:35.935Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 24 reason = Timeout
3613	2024-12-17T21:34:35.935Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
3614	2024-12-17T21:34:35.935Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 24 reason = Timeout
3615	2024-12-17T21:34:35.935Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
3616	2024-12-17T21:34:35.935Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
3617	2024-12-17T21:34:35.935Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 24 reason = Timeout
3618	2024-12-17T21:34:35.935Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
3619	2024-12-17T21:34:35.935Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 70 reason = Timeout
3620	2024-12-17T21:34:35.935Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3621	2024-12-17T21:34:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
3622	2024-12-17T21:34:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
3623	2024-12-17T21:34:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3624	2024-12-17T21:34:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
3625	2024-12-17T21:34:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
3626	2024-12-17T21:34:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
3627	2024-12-17T21:34:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
3628	2024-12-17T21:34:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3629	2024-12-17T21:34:35.937Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1934, tv_nsec: 70546385 } })
3630	2024-12-17T21:34:35.937Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
3631	2024-12-17T21:34:35.938Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3632	2024-12-17T21:34:35.938Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.304477ms iteration = 24 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
3633	2024-12-17T21:34:35.938Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3634	2024-12-17T21:34:35.939Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3635	2024-12-17T21:34:35.939Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3636	2024-12-17T21:34:35.941Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1934, tv_nsec: 74163143 } })
3637	2024-12-17T21:34:35.941Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
3638	2024-12-17T21:34:35.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
3639	2024-12-17T21:34:35.941Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3640	2024-12-17T21:34:35.942Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3641	2024-12-17T21:34:35.942Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
3642	2024-12-17T21:34:35.942Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3643	2024-12-17T21:34:35.942Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3644	2024-12-17T21:34:35.942Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
3645	2024-12-17T21:34:35.942Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3646	2024-12-17T21:34:35.943Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3647	2024-12-17T21:34:35.943Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3648	2024-12-17T21:34:35.943Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3649	2024-12-17T21:34:35.943Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3650	2024-12-17T21:34:35.945Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3651	2024-12-17T21:34:35.946Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
3652	2024-12-17T21:34:35.946Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3653	2024-12-17T21:34:35.946Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
3654	2024-12-17T21:34:35.947Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 11.654436ms iteration = 24 status = {"errors":[],"garbage_collect_requested":[]}
3655	2024-12-17T21:34:35.947Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
3656	2024-12-17T21:34:35.947Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
3657	2024-12-17T21:34:35.947Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
3658	2024-12-17T21:34:35.947Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3659	2024-12-17T21:34:35.948Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
3660	2024-12-17T21:34:35.948Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3661	2024-12-17T21:34:35.950Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3662	2024-12-17T21:34:35.950Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3663	2024-12-17T21:34:35.950Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3664	2024-12-17T21:34:35.950Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3665	2024-12-17T21:34:35.950Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
3666	2024-12-17T21:34:35.950Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
3667	2024-12-17T21:34:35.950Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
3668	2024-12-17T21:34:35.951Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
3669	2024-12-17T21:34:35.951Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3670	2024-12-17T21:34:35.951Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3671	2024-12-17T21:34:35.952Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3672	2024-12-17T21:34:35.952Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1934, tv_nsec: 85384476 } })
3673	2024-12-17T21:34:35.952Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3674	2024-12-17T21:34:35.952Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3675	2024-12-17T21:34:35.952Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
3676	2024-12-17T21:34:35.952Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3677	2024-12-17T21:34:35.953Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3678	2024-12-17T21:34:35.953Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 18.352046ms iteration = 70 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
3679	2024-12-17T21:34:35.955Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3680	2024-12-17T21:34:35.955Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
3681	2024-12-17T21:34:35.955Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3682	2024-12-17T21:34:35.955Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3683	2024-12-17T21:34:35.957Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
3684	2024-12-17T21:34:35.957Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3685	2024-12-17T21:34:35.957Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3686	2024-12-17T21:34:35.957Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3687	2024-12-17T21:34:35.960Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
3688	2024-12-17T21:34:35.960Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3689	2024-12-17T21:34:35.960Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3690	2024-12-17T21:34:35.973Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
3691	2024-12-17T21:34:35.973Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 39.434097ms iteration = 70 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
3692	2024-12-17T21:34:35.979Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
3693	2024-12-17T21:34:35.980Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
3694	2024-12-17T21:34:35.983Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3695	2024-12-17T21:34:35.984Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3696	2024-12-17T21:34:35.984Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
3697	2024-12-17T21:34:35.984Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
3698	2024-12-17T21:34:35.987Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3699	2024-12-17T21:34:35.987Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3700	2024-12-17T21:34:35.988Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
3701	2024-12-17T21:34:35.988Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
3702	2024-12-17T21:34:35.988Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
3703	2024-12-17T21:34:35.988Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
3704	2024-12-17T21:34:35.988Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3705	2024-12-17T21:34:35.989Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 54.30988ms iteration = 24 status = {"errors":[],"records_set_to_done":[]}
3706	2024-12-17T21:34:35.992Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: d3278cd4-2587-4f0f-89dd-b4d4669a6ffe, lookup_type: ById(d3278cd4-2587-4f0f-89dd-b4d4669a6ffe) } result = Ok(())
3707	2024-12-17T21:34:35.992Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3708	2024-12-17T21:34:35.992Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3709	2024-12-17T21:34:35.994Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
3710	2024-12-17T21:34:35.994Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
3711	2024-12-17T21:34:35.995Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 61.304349ms iteration = 24 status = 0
3712	2024-12-17T21:34:36.023Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 87.912699ms iteration = 24 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
3713	2024-12-17T21:34:36.026Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 92.064822ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
3714	2024-12-17T21:34:36.030Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
3715	2024-12-17T21:34:36.030Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
3716	2024-12-17T21:34:36.030Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 95.185476ms iteration = 24 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
3717	2024-12-17T21:34:36.035Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 100.072868ms iteration = 24 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
3718	2024-12-17T21:34:36.059Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 zone_address = fd00:1122:3344:101::2
3719	2024-12-17T21:34:36.059Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3720	2024-12-17T21:34:36.060Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a87df21c-429d-4c9f-8224-3e44cfaeca52", "content-length": "26", "date": "Tue, 17 Dec 2024 21:34:35 GMT"} })
3721	2024-12-17T21:34:36.060Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3722	2024-12-17T21:34:36.062Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
3723	2024-12-17T21:34:36.073Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 139.390782ms iteration = 26 status = {"physical_disks_added":0}
3724	2024-12-17T21:34:36.074Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3725	2024-12-17T21:34:36.074Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3726	2024-12-17T21:34:36.137Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = instance_watcher body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vmms/9c3a9d85-fd07-447b-88fe-34e629fe03c2/state
3727	2024-12-17T21:34:36.141Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
3728	2024-12-17T21:34:36.141Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3729	2024-12-17T21:34:36.141Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb3480dc-6c3e-44e6-873c-ef3e40120197", "content-length": "26", "date": "Tue, 17 Dec 2024 21:34:35 GMT"} })
3730	2024-12-17T21:34:36.141Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3731	2024-12-17T21:34:36.143Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3732	2024-12-17T21:34:36.158Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
3733	2024-12-17T21:34:36.158Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3734	2024-12-17T21:34:36.159Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "eed38c0d-823c-4926-8287-292b9c1a1ebc", "content-length": "26", "date": "Tue, 17 Dec 2024 21:34:35 GMT"} })
3735	2024-12-17T21:34:36.159Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3736	2024-12-17T21:34:36.160Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3737	2024-12-17T21:34:36.171Z	INFO	nexus (BackgroundTasks): VPC route manager sled d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
3738	2024-12-17T21:34:36.171Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
3739	2024-12-17T21:34:36.173Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3740	2024-12-17T21:34:36.175Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
3741	2024-12-17T21:34:36.175Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3742	2024-12-17T21:34:36.175Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3743	2024-12-17T21:34:36.177Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3744	2024-12-17T21:34:36.180Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
3745	2024-12-17T21:34:36.180Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3746	2024-12-17T21:34:36.180Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3747	2024-12-17T21:34:36.191Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = instance_watcher result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vmms/9c3a9d85-fd07-447b-88fe-34e629fe03c2/state", status: 200, headers: {"content-type": "application/json", "x-request-id": "59e85f16-b0db-4e30-b8ff-c010fbf3dc1f", "content-length": "130", "date": "Tue, 17 Dec 2024 21:34:36 GMT"} })
3748	2024-12-17T21:34:36.191Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b52b7e5-9f2d-47af-9aa8-c22ce5f8e068", "content-length": "581", "date": "Tue, 17 Dec 2024 21:34:35 GMT"} })
3749	2024-12-17T21:34:36.191Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3750	2024-12-17T21:34:36.192Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3751	2024-12-17T21:34:36.198Z	DEBG	nexus (BackgroundTasks): updating instance state background_task = instance_watcher instance_id = 447b3daf-2143-44ce-902c-38b7fcb523cc sled_id = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe state = SledVmmState { vmm_state: VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2024-12-17T21:34:30.480209178Z }, migration_in: None, migration_out: None } vmm_id = 9c3a9d85-fd07-447b-88fe-34e629fe03c2
3752	2024-12-17T21:34:36.198Z	INFO	nexus (BackgroundTasks): received new VMM runtime state from sled agent background_task = instance_watcher file = nexus/src/app/instance.rs:2125 instance_id = 447b3daf-2143-44ce-902c-38b7fcb523cc migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 9c3a9d85-fd07-447b-88fe-34e629fe03c2 sled_id = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe vmm_id = 9c3a9d85-fd07-447b-88fe-34e629fe03c2 vmm_state = VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2024-12-17T21:34:30.480209178Z }
3753	2024-12-17T21:34:36.198Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher instance_id = 447b3daf-2143-44ce-902c-38b7fcb523cc roles = RoleSet { roles: {} } sled_id = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe vmm_id = 9c3a9d85-fd07-447b-88fe-34e629fe03c2
3754	2024-12-17T21:34:36.198Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = 447b3daf-2143-44ce-902c-38b7fcb523cc resource = Database result = Ok(()) sled_id = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe vmm_id = 9c3a9d85-fd07-447b-88fe-34e629fe03c2
3755	2024-12-17T21:34:36.214Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 1 total_failed = 0 total_incomplete = 0 total_instances = 1 update_sagas_queued = 0
3756	2024-12-17T21:34:36.214Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {5f521414-80bc-4c28-9d3c-5fece723f202: {10.151.2.176: {0afab819-0aab-48a9-9ca6-4b1a46b348e1}, 10.151.2.175: {0afab819-0aab-48a9-9ca6-4b1a46b348e1}}} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
3757	2024-12-17T21:34:36.214Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
3758	2024-12-17T21:34:36.214Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "7c3a6a5d-3f8a-492a-86d5-e8a4eb26f197", "date": "Tue, 17 Dec 2024 21:34:36 GMT"} })
3759	2024-12-17T21:34:36.214Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3760	2024-12-17T21:34:36.215Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3761	2024-12-17T21:34:36.215Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 280.656296ms iteration = 24 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{"running":1},"pruned_instances":0,"total_instances":1,"update_sagas_queued":0}
3762	2024-12-17T21:34:36.217Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3763	2024-12-17T21:34:36.217Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3764	2024-12-17T21:34:36.220Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3765	2024-12-17T21:34:36.221Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3766	2024-12-17T21:34:36.237Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
3767	2024-12-17T21:34:36.238Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "308a8e41-232a-4fba-af41-458265ddfc7d", "content-length": "1", "date": "Tue, 17 Dec 2024 21:34:35 GMT"} })
3768	2024-12-17T21:34:36.239Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3769	2024-12-17T21:34:36.239Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3770	2024-12-17T21:34:36.254Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 320.35931ms iteration = 24 status = {}
3771	2024-12-17T21:34:36.262Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
3772	2024-12-17T21:34:36.262Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "eb2d03ff-9ec3-488e-b787-f0b1bc38b184", "content-length": "809", "date": "Tue, 17 Dec 2024 21:34:36 GMT"} })
3773	2024-12-17T21:34:36.262Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {VirtualNetworkInterfaceHost { physical_host_ip: fd00:1122:3344:101::1, virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), vni: Vni(385213) }} sled = sock
3774	2024-12-17T21:34:36.262Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
3775	2024-12-17T21:34:36.262Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
3776	2024-12-17T21:34:36.263Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 329.419285ms iteration = 24 status = {}
3777	2024-12-17T21:34:36.265Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 330.735021ms iteration = 24 status = null
3778	2024-12-17T21:34:36.267Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3779	2024-12-17T21:34:36.267Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3780	2024-12-17T21:34:36.269Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3781	2024-12-17T21:34:36.269Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3782	2024-12-17T21:34:36.273Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3783	2024-12-17T21:34:36.273Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3784	2024-12-17T21:34:36.276Z	INFO	nexus (BackgroundTasks): VPC route manager sled d3278cd4-2587-4f0f-89dd-b4d4669a6ffe push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
3785	2024-12-17T21:34:36.276Z	INFO	nexus (BackgroundTasks): VPC route manager sled d3278cd4-2587-4f0f-89dd-b4d4669a6ffe push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
3786	2024-12-17T21:34:36.276Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
3787	2024-12-17T21:34:36.277Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "7c374b33-0744-4153-8c02-421a7d5660ea", "date": "Tue, 17 Dec 2024 21:34:36 GMT"} })
3788	2024-12-17T21:34:36.279Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 344.064552ms iteration = 24 status = {}
3789	2024-12-17T21:34:36.294Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3790	2024-12-17T21:34:36.294Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3791	2024-12-17T21:34:36.296Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 1a3e1a79-c78b-4fb4-921b-c5e1e32e8b40, rack_id: 8216c8d1-8011-434d-a57a-2b1ce2e89ad8, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(d6c2617a-6b89-4779-bc70-3002fb0a7e5e) } rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0
3792	2024-12-17T21:34:36.296Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3793	2024-12-17T21:34:36.297Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3794	2024-12-17T21:34:36.307Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
3795	2024-12-17T21:34:36.307Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "a73bcb92-8d8e-49e3-95fa-a3501ea3e88c", "content-length": "134", "date": "Tue, 17 Dec 2024 21:34:35 GMT"} })
3796	2024-12-17T21:34:36.307Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
3797	2024-12-17T21:34:36.307Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = d6c2617a-6b89-4779-bc70-3002fb0a7e5e
3798	2024-12-17T21:34:36.307Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
3799	2024-12-17T21:34:36.307Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
3800	2024-12-17T21:34:36.307Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3801	2024-12-17T21:34:36.307Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3802	2024-12-17T21:34:36.307Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3803	2024-12-17T21:34:36.307Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
3804	2024-12-17T21:34:36.307Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3805	2024-12-17T21:34:36.307Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
3806	2024-12-17T21:34:36.308Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
3807	2024-12-17T21:34:36.308Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3808	2024-12-17T21:34:36.308Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3809	2024-12-17T21:34:36.310Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
3810	2024-12-17T21:34:36.310Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "66cbf495-cc06-4a28-83a9-d98fc610dd28", "content-length": "2", "date": "Tue, 17 Dec 2024 21:34:35 GMT"} })
3811	2024-12-17T21:34:36.310Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
3812	2024-12-17T21:34:36.310Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "710d04e8-2e26-4153-a4f5-d9ecbb89ecf5", "content-length": "51", "date": "Tue, 17 Dec 2024 21:34:35 GMT"} })
3813	2024-12-17T21:34:36.310Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0
3814	2024-12-17T21:34:36.310Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
3815	2024-12-17T21:34:36.407Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "8c4cd62f-94e5-439f-a60f-0b380f070258", "date": "Tue, 17 Dec 2024 21:34:36 GMT"} })
3816	2024-12-17T21:34:36.407Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch = Switch0
3817	2024-12-17T21:34:36.407Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
3818	2024-12-17T21:34:36.407Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "0fe7e565-197a-498f-9388-d2143539b33c", "content-length": "682", "date": "Tue, 17 Dec 2024 21:34:36 GMT"} })
3819	2024-12-17T21:34:36.407Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3820	2024-12-17T21:34:36.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3821	2024-12-17T21:34:36.410Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3822	2024-12-17T21:34:36.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3823	2024-12-17T21:34:36.414Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3824	2024-12-17T21:34:36.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3825	2024-12-17T21:34:36.416Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3826	2024-12-17T21:34:36.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3827	2024-12-17T21:34:36.423Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 489.59202ms iteration = 24 status = {}
3828	2024-12-17T21:34:40.394Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 369aef73-ff0d-4e01-8c28-3fa8e435866f roles = RoleSet { roles: {} } uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?sort_by=id_ascending
3829	2024-12-17T21:34:40.394Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 369aef73-ff0d-4e01-8c28-3fa8e435866f resource = Database result = Ok(()) uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?sort_by=id_ascending
3830	2024-12-17T21:34:40.397Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3250 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 369aef73-ff0d-4e01-8c28-3fa8e435866f response_code = 200 uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?sort_by=id_ascending
3831	2024-12-17T21:34:40.397Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = f605257a-849f-4a9a-961e-7878f5aaf975 roles = RoleSet { roles: {} } uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMTkzZjYyMi00MGY1LTQyOGYtYmVhMy01MmNjNjdlNzhhMzkifX0%3D
3832	2024-12-17T21:34:40.397Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = f605257a-849f-4a9a-961e-7878f5aaf975 resource = Database result = Ok(()) uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMTkzZjYyMi00MGY1LTQyOGYtYmVhMy01MmNjNjdlNzhhMzkifX0%3D
3833	2024-12-17T21:34:40.399Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2102 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = f605257a-849f-4a9a-961e-7878f5aaf975 response_code = 200 uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMTkzZjYyMi00MGY1LTQyOGYtYmVhMy01MmNjNjdlNzhhMzkifX0%3D
3834	2024-12-17T21:34:40.408Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2588 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = 39cb576a-2bba-4d47-8a59-a02718176d88 response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
3835	2024-12-17T21:34:45.934Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 71 reason = Timeout
3836	2024-12-17T21:34:45.934Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3837	2024-12-17T21:34:45.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3838	2024-12-17T21:34:45.935Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 71 reason = Timeout
3839	2024-12-17T21:34:45.935Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3840	2024-12-17T21:34:45.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3841	2024-12-17T21:34:45.936Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3842	2024-12-17T21:34:45.937Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3843	2024-12-17T21:34:45.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3844	2024-12-17T21:34:45.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3845	2024-12-17T21:34:45.939Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3846	2024-12-17T21:34:45.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3847	2024-12-17T21:34:45.939Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.878802ms iteration = 71 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
3848	2024-12-17T21:34:45.941Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3849	2024-12-17T21:34:45.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3850	2024-12-17T21:34:45.943Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3851	2024-12-17T21:34:45.945Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
3852	2024-12-17T21:34:45.945Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3853	2024-12-17T21:34:45.946Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3854	2024-12-17T21:34:45.954Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
3855	2024-12-17T21:34:45.954Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.439177ms iteration = 71 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
3856	2024-12-17T21:34:50.406Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 308 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = f0599bf0-e0dd-4e09-bce3-00334d12aaef response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
3857	2024-12-17T21:34:55.934Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 72 reason = Timeout
3858	2024-12-17T21:34:55.934Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3859	2024-12-17T21:34:55.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3860	2024-12-17T21:34:55.934Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 72 reason = Timeout
3861	2024-12-17T21:34:55.935Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3862	2024-12-17T21:34:55.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3863	2024-12-17T21:34:55.936Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3864	2024-12-17T21:34:55.937Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3865	2024-12-17T21:34:55.938Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3866	2024-12-17T21:34:55.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3867	2024-12-17T21:34:55.939Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3868	2024-12-17T21:34:55.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3869	2024-12-17T21:34:55.939Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.80345ms iteration = 72 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
3870	2024-12-17T21:34:55.943Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3871	2024-12-17T21:34:55.943Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3872	2024-12-17T21:34:55.945Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3873	2024-12-17T21:34:55.947Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
3874	2024-12-17T21:34:55.947Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3875	2024-12-17T21:34:55.948Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3876	2024-12-17T21:34:55.956Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
3877	2024-12-17T21:34:55.956Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.118479ms iteration = 72 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
3878	2024-12-17T21:35:00.405Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 377 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = 6933f0ec-9fdb-4650-b02e-949c2fc2b39c response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
3879	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 6 reason = Timeout
3880	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 13 reason = Timeout
3881	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
3882	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
3883	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3884	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = external_endpoints iteration = 13 reason = Timeout
3885	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 25 reason = Timeout
3886	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3887	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 13 reason = Timeout
3888	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3889	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 15 reason = Timeout
3890	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 15 reason = Timeout
3891	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 25 reason = Timeout
3892	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3893	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 14 reason = Timeout
3894	2024-12-17T21:35:05.934Z	WARN	nexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5
3895	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 18.551\u{b5}s iteration = 14 status = {"enabled":false,"target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5"}
3896	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 13 reason = Timeout
3897	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 27 reason = Timeout
3898	2024-12-17T21:35:05.934Z	INFO	nexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-17 21:25:05.934683520 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
3899	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 13 reason = Timeout
3900	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
3901	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
3902	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3903	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3904	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 25 reason = Timeout
3905	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
3906	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 25 reason = Timeout
3907	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 13 reason = Timeout
3908	2024-12-17T21:35:05.934Z	INFO	nexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
3909	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
3910	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 25 reason = Timeout
3911	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 73 reason = Timeout
3912	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
3913	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 13 reason = Timeout
3914	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3915	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 25 reason = Timeout
3916	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 25 reason = Timeout
3917	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3918	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3919	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 25 reason = Timeout
3920	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
3921	2024-12-17T21:35:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 13 reason = Timeout
3922	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
3923	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 13 reason = Timeout
3924	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
3925	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 13 reason = Timeout
3926	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
3927	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 25 reason = Timeout
3928	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 25 reason = Timeout
3929	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
3930	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
3931	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 25 reason = Timeout
3932	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
3933	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 25 reason = Timeout
3934	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
3935	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 25 reason = Timeout
3936	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
3937	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
3938	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 25 reason = Timeout
3939	2024-12-17T21:35:05.935Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
3940	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3941	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 73 reason = Timeout
3942	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3943	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3944	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3945	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3946	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
3947	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
3948	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
3949	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
3950	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3951	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3952	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
3953	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3954	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3955	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3956	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3957	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
3958	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
3959	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
3960	2024-12-17T21:35:05.935Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("5dff2793-3ce8-4251-ba71-1a446966cbf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a761a47e-08b9-41c4-9bd8-8d77651dfa62.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a761a47e-08b9-41c4-9bd8-8d77651dfa62.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1964, tv_nsec: 68638973 } })
3961	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
3962	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
3963	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
3964	2024-12-17T21:35:05.935Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0005c310-624c-4312-b0fd-35f571746851.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6ac4d5e8-fcd9-4aa9-aed5-fb3dc293aac2.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9b804364-993d-47ba-81a1-d0ef32f96529.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("9b804364-993d-47ba-81a1-d0ef32f96529.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1964, tv_nsec: 68668794 } })
3965	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
3966	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
3967	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
3968	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
3969	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
3970	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3971	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3972	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
3973	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
3974	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
3975	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
3976	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
3977	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
3978	2024-12-17T21:35:05.935Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.182792ms iteration = 13 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
3979	2024-12-17T21:35:05.936Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
3980	2024-12-17T21:35:05.936Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.413588ms iteration = 13 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
3981	2024-12-17T21:35:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
3982	2024-12-17T21:35:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3983	2024-12-17T21:35:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3984	2024-12-17T21:35:05.937Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.250358ms iteration = 25 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
3985	2024-12-17T21:35:05.938Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3986	2024-12-17T21:35:05.938Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3987	2024-12-17T21:35:05.939Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3988	2024-12-17T21:35:05.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3989	2024-12-17T21:35:05.940Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1964, tv_nsec: 73320880 } })
3990	2024-12-17T21:35:05.940Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
3991	2024-12-17T21:35:05.940Z	INFO	nexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-17 21:25:05.934683520 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
3992	2024-12-17T21:35:05.940Z	DEBG	nexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 5.983352ms iteration = 13 status = {"expiration":"2024-12-17T21:25:05.934683520Z","pruned":[]}
3993	2024-12-17T21:35:05.940Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
3994	2024-12-17T21:35:05.940Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3995	2024-12-17T21:35:05.940Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3996	2024-12-17T21:35:05.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
3997	2024-12-17T21:35:05.941Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
3998	2024-12-17T21:35:05.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
3999	2024-12-17T21:35:05.942Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4000	2024-12-17T21:35:05.943Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4001	2024-12-17T21:35:05.943Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1964, tv_nsec: 76967199 } })
4002	2024-12-17T21:35:05.944Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
4003	2024-12-17T21:35:05.944Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4004	2024-12-17T21:35:05.945Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
4005	2024-12-17T21:35:05.945Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4006	2024-12-17T21:35:05.945Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4007	2024-12-17T21:35:05.946Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
4008	2024-12-17T21:35:05.946Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4009	2024-12-17T21:35:05.946Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4010	2024-12-17T21:35:05.946Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
4011	2024-12-17T21:35:05.946Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4012	2024-12-17T21:35:05.946Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4013	2024-12-17T21:35:05.947Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4014	2024-12-17T21:35:05.947Z	DEBG	nexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 12.391275ms iteration = 13 status = {"errors":[],"found_port_ok":[]}
4015	2024-12-17T21:35:05.947Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4016	2024-12-17T21:35:05.947Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4017	2024-12-17T21:35:05.950Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4018	2024-12-17T21:35:05.950Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4019	2024-12-17T21:35:05.950Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4020	2024-12-17T21:35:05.953Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 18.441329ms iteration = 25 status = {"errors":[],"garbage_collect_requested":[]}
4021	2024-12-17T21:35:05.954Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4022	2024-12-17T21:35:05.957Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
4023	2024-12-17T21:35:05.957Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
4024	2024-12-17T21:35:05.957Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
4025	2024-12-17T21:35:05.959Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4026	2024-12-17T21:35:05.961Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
4027	2024-12-17T21:35:05.961Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
4028	2024-12-17T21:35:05.962Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
4029	2024-12-17T21:35:05.969Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4030	2024-12-17T21:35:05.969Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4031	2024-12-17T21:35:05.973Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1964, tv_nsec: 106346304 } })
4032	2024-12-17T21:35:05.973Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4033	2024-12-17T21:35:05.973Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
4034	2024-12-17T21:35:05.973Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4035	2024-12-17T21:35:05.975Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4036	2024-12-17T21:35:05.975Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4037	2024-12-17T21:35:05.977Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4038	2024-12-17T21:35:05.977Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4039	2024-12-17T21:35:05.988Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4040	2024-12-17T21:35:05.991Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
4041	2024-12-17T21:35:05.991Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
4042	2024-12-17T21:35:05.992Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
4043	2024-12-17T21:35:06.001Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4044	2024-12-17T21:35:06.003Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 69.042938ms iteration = 25 status = {"errors":[],"records_set_to_done":[]}
4045	2024-12-17T21:35:06.004Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
4046	2024-12-17T21:35:06.004Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4047	2024-12-17T21:35:06.004Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4048	2024-12-17T21:35:06.006Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4049	2024-12-17T21:35:06.006Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4050	2024-12-17T21:35:06.006Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4051	2024-12-17T21:35:06.008Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
4052	2024-12-17T21:35:06.008Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4053	2024-12-17T21:35:06.008Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4054	2024-12-17T21:35:06.009Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4055	2024-12-17T21:35:06.009Z	DEBG	nexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
4056	2024-12-17T21:35:06.009Z	INFO	nexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
4057	2024-12-17T21:35:06.009Z	DEBG	nexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 74.460825ms iteration = 13 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
4058	2024-12-17T21:35:06.009Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
4059	2024-12-17T21:35:06.010Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
4060	2024-12-17T21:35:06.010Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
4061	2024-12-17T21:35:06.010Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
4062	2024-12-17T21:35:06.010Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
4063	2024-12-17T21:35:06.010Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
4064	2024-12-17T21:35:06.010Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
4065	2024-12-17T21:35:06.010Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4066	2024-12-17T21:35:06.010Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
4067	2024-12-17T21:35:06.011Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
4068	2024-12-17T21:35:06.011Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
4069	2024-12-17T21:35:06.012Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
4070	2024-12-17T21:35:06.012Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
4071	2024-12-17T21:35:06.012Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
4072	2024-12-17T21:35:06.012Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
4073	2024-12-17T21:35:06.013Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4074	2024-12-17T21:35:06.014Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4075	2024-12-17T21:35:06.014Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4076	2024-12-17T21:35:06.017Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
4077	2024-12-17T21:35:06.017Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4078	2024-12-17T21:35:06.017Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
4079	2024-12-17T21:35:06.017Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
4080	2024-12-17T21:35:06.017Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4081	2024-12-17T21:35:06.017Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4082	2024-12-17T21:35:06.017Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
4083	2024-12-17T21:35:06.017Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
4084	2024-12-17T21:35:06.017Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4085	2024-12-17T21:35:06.017Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4086	2024-12-17T21:35:06.017Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
4087	2024-12-17T21:35:06.018Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
4088	2024-12-17T21:35:06.020Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
4089	2024-12-17T21:35:06.020Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
4090	2024-12-17T21:35:06.020Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
4091	2024-12-17T21:35:06.020Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
4092	2024-12-17T21:35:06.021Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
4093	2024-12-17T21:35:06.021Z	DEBG	nexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 86.916712ms iteration = 6 status = {"nsuccess":5}
4094	2024-12-17T21:35:06.022Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4095	2024-12-17T21:35:06.022Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4096	2024-12-17T21:35:06.024Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
4097	2024-12-17T21:35:06.024Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4098	2024-12-17T21:35:06.024Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4099	2024-12-17T21:35:06.027Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
4100	2024-12-17T21:35:06.030Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
4101	2024-12-17T21:35:06.030Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 95.784532ms iteration = 73 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
4102	2024-12-17T21:35:06.032Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4103	2024-12-17T21:35:06.033Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4104	2024-12-17T21:35:06.034Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
4105	2024-12-17T21:35:06.035Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
4106	2024-12-17T21:35:06.035Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4107	2024-12-17T21:35:06.035Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4108	2024-12-17T21:35:06.037Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4109	2024-12-17T21:35:06.041Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: d3278cd4-2587-4f0f-89dd-b4d4669a6ffe, lookup_type: ById(d3278cd4-2587-4f0f-89dd-b4d4669a6ffe) } result = Ok(())
4110	2024-12-17T21:35:06.041Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4111	2024-12-17T21:35:06.041Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4112	2024-12-17T21:35:06.044Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 109.36565ms iteration = 25 status = 0
4113	2024-12-17T21:35:06.057Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 122.038092ms iteration = 73 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
4114	2024-12-17T21:35:06.057Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
4115	2024-12-17T21:35:06.057Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
4116	2024-12-17T21:35:06.057Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
4117	2024-12-17T21:35:06.057Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4118	2024-12-17T21:35:06.057Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4119	2024-12-17T21:35:06.058Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
4120	2024-12-17T21:35:06.058Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
4121	2024-12-17T21:35:06.058Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 123.301427ms iteration = 25 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
4122	2024-12-17T21:35:06.058Z	DEBG	nexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
4123	2024-12-17T21:35:06.058Z	INFO	nexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
4124	2024-12-17T21:35:06.058Z	DEBG	nexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 124.171031ms iteration = 13 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
4125	2024-12-17T21:35:06.058Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4126	2024-12-17T21:35:06.059Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
4127	2024-12-17T21:35:06.059Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
4128	2024-12-17T21:35:06.060Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 125.225949ms iteration = 25 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
4129	2024-12-17T21:35:06.061Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 126.169245ms iteration = 25 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
4130	2024-12-17T21:35:06.062Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4131	2024-12-17T21:35:06.062Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4132	2024-12-17T21:35:06.062Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
4133	2024-12-17T21:35:06.062Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4134	2024-12-17T21:35:06.063Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4135	2024-12-17T21:35:06.063Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4136	2024-12-17T21:35:06.064Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 1 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
4137	2024-12-17T21:35:06.064Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 129.177765ms iteration = 25 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":1,"total_instances":0,"update_sagas_queued":0}
4138	2024-12-17T21:35:06.066Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
4139	2024-12-17T21:35:06.066Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4140	2024-12-17T21:35:06.066Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4141	2024-12-17T21:35:06.066Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 131.683374ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
4142	2024-12-17T21:35:06.069Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
4143	2024-12-17T21:35:06.069Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
4144	2024-12-17T21:35:06.069Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
4145	2024-12-17T21:35:06.069Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4146	2024-12-17T21:35:06.070Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4147	2024-12-17T21:35:06.071Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4148	2024-12-17T21:35:06.074Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
4149	2024-12-17T21:35:06.074Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4150	2024-12-17T21:35:06.074Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4151	2024-12-17T21:35:06.081Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
4152	2024-12-17T21:35:06.097Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
4153	2024-12-17T21:35:06.097Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = ea6acdde-77d4-4bf6-b578-d9a0a29ac37b dns_zone_name = oxide.test found_so_far = 0 version = 2
4154	2024-12-17T21:35:06.097Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4155	2024-12-17T21:35:06.098Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4156	2024-12-17T21:35:06.100Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4157	2024-12-17T21:35:06.104Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
4158	2024-12-17T21:35:06.104Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4159	2024-12-17T21:35:06.104Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4160	2024-12-17T21:35:06.106Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = ea6acdde-77d4-4bf6-b578-d9a0a29ac37b dns_zone_name = oxide.test found_so_far = 1 version = 2
4161	2024-12-17T21:35:06.106Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
4162	2024-12-17T21:35:06.106Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-17 21:23:49.491999 UTC dns_group = external generation = 2
4163	2024-12-17T21:35:06.106Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 171.704816ms iteration = 13 status = {"generation":2}
4164	2024-12-17T21:35:06.109Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
4165	2024-12-17T21:35:06.109Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 574f0c35-b8f2-48fe-831c-78afc4a62aaa dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
4166	2024-12-17T21:35:06.109Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4167	2024-12-17T21:35:06.109Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4168	2024-12-17T21:35:06.111Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4169	2024-12-17T21:35:06.113Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4170	2024-12-17T21:35:06.113Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4171	2024-12-17T21:35:06.115Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4172	2024-12-17T21:35:06.116Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
4173	2024-12-17T21:35:06.116Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4174	2024-12-17T21:35:06.116Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4175	2024-12-17T21:35:06.116Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
4176	2024-12-17T21:35:06.116Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4177	2024-12-17T21:35:06.117Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4178	2024-12-17T21:35:06.118Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 574f0c35-b8f2-48fe-831c-78afc4a62aaa dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
4179	2024-12-17T21:35:06.118Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
4180	2024-12-17T21:35:06.118Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-17 21:23:48.446255 UTC dns_group = internal generation = 1
4181	2024-12-17T21:35:06.118Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 183.683121ms iteration = 13 status = {"generation":1}
4182	2024-12-17T21:35:06.118Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4183	2024-12-17T21:35:06.119Z	INFO	nexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-17T21:23:03.821125Z, time_modified: 2024-12-17T21:23:03.821125Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 81e78e1e-e55a-44f9-abdc-c9c916e1d9fd, db_silo: Silo { identity: SiloIdentity { id: 81e78e1e-e55a-44f9-abdc-c9c916e1d9fd, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-17T21:23:49.443930Z, time_modified: 2024-12-17T21:23:49.443930Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "e275643b21ab02733b9616c9cd3b0193e27c7f6294b7d310d6f1726c72768e50" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 81e78e1e-e55a-44f9-abdc-c9c916e1d9fd, db_silo: Silo { identity: SiloIdentity { id: 81e78e1e-e55a-44f9-abdc-c9c916e1d9fd, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-17T21:23:49.443930Z, time_modified: 2024-12-17T21:23:49.443930Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "e275643b21ab02733b9616c9cd3b0193e27c7f6294b7d310d6f1726c72768e50" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:94
4184	2024-12-17T21:35:06.119Z	DEBG	nexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 185.387057ms iteration = 13 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"81e78e1e-e55a-44f9-abdc-c9c916e1d9fd","tls_certs":["e275643b21ab02733b9616c9cd3b0193e27c7f6294b7d310d6f1726c72768e50"]}},"default_endpoint":{"silo_id":"81e78e1e-e55a-44f9-abdc-c9c916e1d9fd","tls_certs":["e275643b21ab02733b9616c9cd3b0193e27c7f6294b7d310d6f1726c72768e50"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
4185	2024-12-17T21:35:06.120Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
4186	2024-12-17T21:35:06.120Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4187	2024-12-17T21:35:06.120Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4188	2024-12-17T21:35:06.141Z	WARN	nexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
4189	2024-12-17T21:35:06.141Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 206.800877ms iteration = 13 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
4190	2024-12-17T21:35:06.146Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 211.856533ms iteration = 27 status = {"physical_disks_added":0}
4191	2024-12-17T21:35:06.200Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
4192	2024-12-17T21:35:06.200Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4193	2024-12-17T21:35:06.201Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d640c9e5-c7ed-4679-863b-876ffdc1dd4f", "content-length": "26", "date": "Tue, 17 Dec 2024 21:35:05 GMT"} })
4194	2024-12-17T21:35:06.201Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4195	2024-12-17T21:35:06.202Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
4196	2024-12-17T21:35:06.203Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
4197	2024-12-17T21:35:06.283Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 zone_address = fd00:1122:3344:101::2
4198	2024-12-17T21:35:06.283Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4199	2024-12-17T21:35:06.283Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "efbd6774-ecf3-4897-a1d6-5870e921b770", "content-length": "26", "date": "Tue, 17 Dec 2024 21:35:05 GMT"} })
4200	2024-12-17T21:35:06.283Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4201	2024-12-17T21:35:06.285Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
4202	2024-12-17T21:35:06.314Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
4203	2024-12-17T21:35:06.314Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4204	2024-12-17T21:35:06.314Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "da0484f2-320b-43fa-adef-5af6405dcc6c", "content-length": "26", "date": "Tue, 17 Dec 2024 21:35:05 GMT"} })
4205	2024-12-17T21:35:06.314Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4206	2024-12-17T21:35:06.316Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
4207	2024-12-17T21:35:06.319Z	INFO	nexus (BackgroundTasks): VPC route manager sled d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
4208	2024-12-17T21:35:06.319Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
4209	2024-12-17T21:35:06.319Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "053223d6-236b-40bf-a343-abb00b87a5ba", "content-length": "372", "date": "Tue, 17 Dec 2024 21:35:06 GMT"} })
4210	2024-12-17T21:35:06.319Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4211	2024-12-17T21:35:06.320Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4212	2024-12-17T21:35:06.320Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
4213	2024-12-17T21:35:06.321Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "90f201db-8101-4316-86f4-137f8264e77e", "date": "Tue, 17 Dec 2024 21:35:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4214	2024-12-17T21:35:06.321Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4215	2024-12-17T21:35:06.322Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "ae9de963-2860-4e98-b213-cc56cb7a6dcc", "date": "Tue, 17 Dec 2024 21:35:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4216	2024-12-17T21:35:06.322Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4217	2024-12-17T21:35:06.324Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4218	2024-12-17T21:35:06.324Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 390.300843ms iteration = 15 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
4219	2024-12-17T21:35:06.367Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
4220	2024-12-17T21:35:06.368Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
4221	2024-12-17T21:35:06.410Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
4222	2024-12-17T21:35:06.422Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "34e68cd9-ca1a-4e43-881e-577f46f6e466", "date": "Tue, 17 Dec 2024 21:35:06 GMT"} })
4223	2024-12-17T21:35:06.422Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4224	2024-12-17T21:35:06.423Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4225	2024-12-17T21:35:06.425Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4226	2024-12-17T21:35:06.425Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4227	2024-12-17T21:35:06.428Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4228	2024-12-17T21:35:06.428Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4229	2024-12-17T21:35:06.438Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 503.704302ms iteration = 25 status = {}
4230	2024-12-17T21:35:06.450Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
4231	2024-12-17T21:35:06.451Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "a0660b3a-6f46-4204-823e-2b3b0956a4f2", "content-length": "691", "date": "Tue, 17 Dec 2024 21:35:06 GMT"} })
4232	2024-12-17T21:35:06.451Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
4233	2024-12-17T21:35:06.451Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
4234	2024-12-17T21:35:06.451Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
4235	2024-12-17T21:35:06.453Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 518.818491ms iteration = 25 status = {}
4236	2024-12-17T21:35:06.455Z	INFO	nexus (BackgroundTasks): VPC route manager sled d3278cd4-2587-4f0f-89dd-b4d4669a6ffe push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
4237	2024-12-17T21:35:06.455Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
4238	2024-12-17T21:35:06.455Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "6486a343-febb-473f-8ccd-06fc724af40e", "date": "Tue, 17 Dec 2024 21:35:06 GMT"} })
4239	2024-12-17T21:35:06.458Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 523.139148ms iteration = 25 status = {}
4240	2024-12-17T21:35:06.463Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
4241	2024-12-17T21:35:06.463Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "36aebd3c-66d7-4ce8-a321-a43d876429e9", "content-length": "2", "date": "Tue, 17 Dec 2024 21:35:05 GMT"} })
4242	2024-12-17T21:35:06.464Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4243	2024-12-17T21:35:06.464Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4244	2024-12-17T21:35:06.467Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 532.4858ms iteration = 25 status = null
4245	2024-12-17T21:35:06.499Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
4246	2024-12-17T21:35:06.499Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "f3e70940-ed4d-4ddc-9d24-909f6091d796", "date": "Tue, 17 Dec 2024 21:35:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4247	2024-12-17T21:35:06.499Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4248	2024-12-17T21:35:06.501Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "deb58759-01a4-475a-9fcc-2ca5db1a5871", "date": "Tue, 17 Dec 2024 21:35:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4249	2024-12-17T21:35:06.501Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4250	2024-12-17T21:35:06.503Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "1fc9cc5c-8bd6-4578-8af4-70b1be5d9616", "date": "Tue, 17 Dec 2024 21:35:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4251	2024-12-17T21:35:06.503Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4252	2024-12-17T21:35:06.505Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4253	2024-12-17T21:35:06.505Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 570.943702ms iteration = 15 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
4254	2024-12-17T21:35:06.549Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4255	2024-12-17T21:35:06.550Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4256	2024-12-17T21:35:06.552Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 1a3e1a79-c78b-4fb4-921b-c5e1e32e8b40, rack_id: 8216c8d1-8011-434d-a57a-2b1ce2e89ad8, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(d6c2617a-6b89-4779-bc70-3002fb0a7e5e) } rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0
4257	2024-12-17T21:35:06.552Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4258	2024-12-17T21:35:06.552Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4259	2024-12-17T21:35:06.561Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
4260	2024-12-17T21:35:06.562Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "39100467-b1d8-4410-a583-9576f988d22f", "content-length": "134", "date": "Tue, 17 Dec 2024 21:35:06 GMT"} })
4261	2024-12-17T21:35:06.562Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
4262	2024-12-17T21:35:06.562Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = d6c2617a-6b89-4779-bc70-3002fb0a7e5e
4263	2024-12-17T21:35:06.562Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
4264	2024-12-17T21:35:06.562Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
4265	2024-12-17T21:35:06.562Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4266	2024-12-17T21:35:06.562Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4267	2024-12-17T21:35:06.562Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4268	2024-12-17T21:35:06.562Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
4269	2024-12-17T21:35:06.562Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4270	2024-12-17T21:35:06.562Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
4271	2024-12-17T21:35:06.562Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
4272	2024-12-17T21:35:06.562Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4273	2024-12-17T21:35:06.563Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4274	2024-12-17T21:35:06.565Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
4275	2024-12-17T21:35:06.565Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "cf03b5b8-18d4-4f22-ac61-ecef4a7b539c", "content-length": "2", "date": "Tue, 17 Dec 2024 21:35:06 GMT"} })
4276	2024-12-17T21:35:06.565Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
4277	2024-12-17T21:35:06.565Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "3bf4d3cb-b907-4509-b550-5e9ec35d4ced", "content-length": "51", "date": "Tue, 17 Dec 2024 21:35:06 GMT"} })
4278	2024-12-17T21:35:06.565Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0
4279	2024-12-17T21:35:06.565Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
4280	2024-12-17T21:35:06.650Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "ef337cef-6eed-4dc6-b05f-c9ddaa6b2f0a", "date": "Tue, 17 Dec 2024 21:35:06 GMT"} })
4281	2024-12-17T21:35:06.650Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch = Switch0
4282	2024-12-17T21:35:06.650Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
4283	2024-12-17T21:35:06.650Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c14bbe0-b3ca-45bd-af39-63c9912dae18", "content-length": "682", "date": "Tue, 17 Dec 2024 21:35:06 GMT"} })
4284	2024-12-17T21:35:06.650Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4285	2024-12-17T21:35:06.650Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4286	2024-12-17T21:35:06.654Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4287	2024-12-17T21:35:06.654Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4288	2024-12-17T21:35:06.657Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4289	2024-12-17T21:35:06.657Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4290	2024-12-17T21:35:06.659Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4291	2024-12-17T21:35:06.659Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4292	2024-12-17T21:35:06.666Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 732.169905ms iteration = 25 status = {}
4293	2024-12-17T21:35:10.394Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 49bd6bcc-f661-4826-8d4b-d074b1392168 roles = RoleSet { roles: {} } uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?sort_by=id_ascending
4294	2024-12-17T21:35:10.394Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 49bd6bcc-f661-4826-8d4b-d074b1392168 resource = Database result = Ok(()) uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?sort_by=id_ascending
4295	2024-12-17T21:35:10.397Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3662 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 49bd6bcc-f661-4826-8d4b-d074b1392168 response_code = 200 uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?sort_by=id_ascending
4296	2024-12-17T21:35:10.397Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 67be50a3-2950-4177-b771-690736074e54 roles = RoleSet { roles: {} } uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMTkzZjYyMi00MGY1LTQyOGYtYmVhMy01MmNjNjdlNzhhMzkifX0%3D
4297	2024-12-17T21:35:10.398Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 67be50a3-2950-4177-b771-690736074e54 resource = Database result = Ok(()) uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMTkzZjYyMi00MGY1LTQyOGYtYmVhMy01MmNjNjdlNzhhMzkifX0%3D
4298	2024-12-17T21:35:10.399Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2153 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 67be50a3-2950-4177-b771-690736074e54 response_code = 200 uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMTkzZjYyMi00MGY1LTQyOGYtYmVhMy01MmNjNjdlNzhhMzkifX0%3D
4299	2024-12-17T21:35:10.405Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 404 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = 72408690-3288-4ced-b248-d9b86bd6bc73 response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
4300	2024-12-17T21:35:15.933Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 74 reason = Timeout
4301	2024-12-17T21:35:15.933Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4302	2024-12-17T21:35:15.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4303	2024-12-17T21:35:15.935Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 74 reason = Timeout
4304	2024-12-17T21:35:15.935Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4305	2024-12-17T21:35:15.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4306	2024-12-17T21:35:15.936Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4307	2024-12-17T21:35:15.937Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4308	2024-12-17T21:35:15.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4309	2024-12-17T21:35:15.938Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4310	2024-12-17T21:35:15.938Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4311	2024-12-17T21:35:15.938Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4312	2024-12-17T21:35:15.940Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4313	2024-12-17T21:35:15.940Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4314	2024-12-17T21:35:15.942Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4315	2024-12-17T21:35:15.944Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
4316	2024-12-17T21:35:15.944Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4317	2024-12-17T21:35:15.944Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4318	2024-12-17T21:35:15.956Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
4319	2024-12-17T21:35:15.956Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.104405ms iteration = 74 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
4320	2024-12-17T21:35:15.966Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.527174ms iteration = 74 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
4321	2024-12-17T21:35:20.405Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 436 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = bf28028a-5201-45b4-961d-800b86184fbb response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
4322	2024-12-17T21:35:25.934Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 75 reason = Timeout
4323	2024-12-17T21:35:25.934Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4324	2024-12-17T21:35:25.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4325	2024-12-17T21:35:25.935Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 75 reason = Timeout
4326	2024-12-17T21:35:25.935Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4327	2024-12-17T21:35:25.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4328	2024-12-17T21:35:25.936Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4329	2024-12-17T21:35:25.937Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4330	2024-12-17T21:35:25.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4331	2024-12-17T21:35:25.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4332	2024-12-17T21:35:25.939Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4333	2024-12-17T21:35:25.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4334	2024-12-17T21:35:25.940Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4335	2024-12-17T21:35:25.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4336	2024-12-17T21:35:25.942Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4337	2024-12-17T21:35:25.944Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
4338	2024-12-17T21:35:25.944Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4339	2024-12-17T21:35:25.945Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4340	2024-12-17T21:35:25.955Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
4341	2024-12-17T21:35:25.955Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.374759ms iteration = 75 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
4342	2024-12-17T21:35:25.965Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 30.432503ms iteration = 75 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
4343	2024-12-17T21:35:30.405Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 329 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = 6ae61642-6c92-422d-9356-eca4440183be response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
4344	2024-12-17T21:35:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 76 reason = Timeout
4345	2024-12-17T21:35:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 26 reason = Timeout
4346	2024-12-17T21:35:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4347	2024-12-17T21:35:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 26 reason = Timeout
4348	2024-12-17T21:35:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4349	2024-12-17T21:35:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4350	2024-12-17T21:35:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 26 reason = Timeout
4351	2024-12-17T21:35:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 26 reason = Timeout
4352	2024-12-17T21:35:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
4353	2024-12-17T21:35:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 26 reason = Timeout
4354	2024-12-17T21:35:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4355	2024-12-17T21:35:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 26 reason = Timeout
4356	2024-12-17T21:35:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 26 reason = Timeout
4357	2024-12-17T21:35:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
4358	2024-12-17T21:35:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 28 reason = Timeout
4359	2024-12-17T21:35:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4360	2024-12-17T21:35:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4361	2024-12-17T21:35:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 26 reason = Timeout
4362	2024-12-17T21:35:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
4363	2024-12-17T21:35:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 26 reason = Timeout
4364	2024-12-17T21:35:35.934Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
4365	2024-12-17T21:35:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4366	2024-12-17T21:35:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 26 reason = Timeout
4367	2024-12-17T21:35:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
4368	2024-12-17T21:35:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 26 reason = Timeout
4369	2024-12-17T21:35:35.934Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
4370	2024-12-17T21:35:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
4371	2024-12-17T21:35:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 76 reason = Timeout
4372	2024-12-17T21:35:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4373	2024-12-17T21:35:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 26 reason = Timeout
4374	2024-12-17T21:35:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 26 reason = Timeout
4375	2024-12-17T21:35:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
4376	2024-12-17T21:35:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
4377	2024-12-17T21:35:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 26 reason = Timeout
4378	2024-12-17T21:35:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
4379	2024-12-17T21:35:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4380	2024-12-17T21:35:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
4381	2024-12-17T21:35:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4382	2024-12-17T21:35:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4383	2024-12-17T21:35:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
4384	2024-12-17T21:35:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4385	2024-12-17T21:35:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4386	2024-12-17T21:35:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4387	2024-12-17T21:35:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4388	2024-12-17T21:35:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
4389	2024-12-17T21:35:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
4390	2024-12-17T21:35:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4391	2024-12-17T21:35:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
4392	2024-12-17T21:35:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
4393	2024-12-17T21:35:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
4394	2024-12-17T21:35:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
4395	2024-12-17T21:35:35.937Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4396	2024-12-17T21:35:35.938Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4397	2024-12-17T21:35:35.938Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1994, tv_nsec: 71976673 } })
4398	2024-12-17T21:35:35.938Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1994, tv_nsec: 72114157 } })
4399	2024-12-17T21:35:35.938Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
4400	2024-12-17T21:35:35.938Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4401	2024-12-17T21:35:35.938Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
4402	2024-12-17T21:35:35.938Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.350088ms iteration = 26 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
4403	2024-12-17T21:35:35.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4404	2024-12-17T21:35:35.941Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4405	2024-12-17T21:35:35.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4406	2024-12-17T21:35:35.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
4407	2024-12-17T21:35:35.941Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4408	2024-12-17T21:35:35.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4409	2024-12-17T21:35:35.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
4410	2024-12-17T21:35:35.941Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4411	2024-12-17T21:35:35.942Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4412	2024-12-17T21:35:35.943Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4413	2024-12-17T21:35:35.944Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4414	2024-12-17T21:35:35.945Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 10.442633ms iteration = 26 status = {"errors":[],"records_set_to_done":[]}
4415	2024-12-17T21:35:35.945Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4416	2024-12-17T21:35:35.945Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4417	2024-12-17T21:35:35.946Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
4418	2024-12-17T21:35:35.946Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4419	2024-12-17T21:35:35.946Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4420	2024-12-17T21:35:35.946Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
4421	2024-12-17T21:35:35.946Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4422	2024-12-17T21:35:35.946Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4423	2024-12-17T21:35:35.948Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4424	2024-12-17T21:35:35.949Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4425	2024-12-17T21:35:35.949Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4426	2024-12-17T21:35:35.949Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4427	2024-12-17T21:35:35.950Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4428	2024-12-17T21:35:35.950Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4429	2024-12-17T21:35:35.950Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
4430	2024-12-17T21:35:35.950Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
4431	2024-12-17T21:35:35.950Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4432	2024-12-17T21:35:35.950Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
4433	2024-12-17T21:35:35.953Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4434	2024-12-17T21:35:35.954Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4435	2024-12-17T21:35:35.954Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1994, tv_nsec: 87969216 } })
4436	2024-12-17T21:35:35.954Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4437	2024-12-17T21:35:35.954Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
4438	2024-12-17T21:35:35.956Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
4439	2024-12-17T21:35:35.956Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4440	2024-12-17T21:35:35.956Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4441	2024-12-17T21:35:35.957Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
4442	2024-12-17T21:35:35.957Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4443	2024-12-17T21:35:35.957Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4444	2024-12-17T21:35:35.960Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
4445	2024-12-17T21:35:35.960Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4446	2024-12-17T21:35:35.960Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4447	2024-12-17T21:35:35.977Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
4448	2024-12-17T21:35:35.977Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 42.700946ms iteration = 76 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
4449	2024-12-17T21:35:35.978Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4450	2024-12-17T21:35:35.979Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
4451	2024-12-17T21:35:35.979Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4452	2024-12-17T21:35:35.979Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
4453	2024-12-17T21:35:35.984Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
4454	2024-12-17T21:35:35.984Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
4455	2024-12-17T21:35:35.986Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 51.260327ms iteration = 26 status = {"errors":[],"garbage_collect_requested":[]}
4456	2024-12-17T21:35:35.986Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 51.74717ms iteration = 76 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
4457	2024-12-17T21:35:35.988Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
4458	2024-12-17T21:35:35.988Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
4459	2024-12-17T21:35:35.990Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
4460	2024-12-17T21:35:35.990Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
4461	2024-12-17T21:35:35.990Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
4462	2024-12-17T21:35:35.990Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
4463	2024-12-17T21:35:35.998Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
4464	2024-12-17T21:35:35.998Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 63.580341ms iteration = 26 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
4465	2024-12-17T21:35:36.032Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
4466	2024-12-17T21:35:36.032Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4467	2024-12-17T21:35:36.032Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "94db6e4c-4e21-4756-9c67-076e8635a9d0", "content-length": "26", "date": "Tue, 17 Dec 2024 21:35:36 GMT"} })
4468	2024-12-17T21:35:36.032Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4469	2024-12-17T21:35:36.034Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
4470	2024-12-17T21:35:36.039Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
4471	2024-12-17T21:35:36.039Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4472	2024-12-17T21:35:36.039Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3c2b88a2-bb8c-4781-8b7c-01f1d0eed5b6", "content-length": "26", "date": "Tue, 17 Dec 2024 21:35:36 GMT"} })
4473	2024-12-17T21:35:36.039Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4474	2024-12-17T21:35:36.041Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
4475	2024-12-17T21:35:36.049Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4476	2024-12-17T21:35:36.049Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4477	2024-12-17T21:35:36.056Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 zone_address = fd00:1122:3344:101::2
4478	2024-12-17T21:35:36.056Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4479	2024-12-17T21:35:36.057Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8dfd1d31-916f-4ede-98a9-b524134753df", "content-length": "26", "date": "Tue, 17 Dec 2024 21:35:36 GMT"} })
4480	2024-12-17T21:35:36.057Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4481	2024-12-17T21:35:36.058Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
4482	2024-12-17T21:35:36.067Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 132.580298ms iteration = 28 status = {"physical_disks_added":0}
4483	2024-12-17T21:35:36.106Z	INFO	nexus (BackgroundTasks): VPC route manager sled d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
4484	2024-12-17T21:35:36.106Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
4485	2024-12-17T21:35:36.108Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4486	2024-12-17T21:35:36.109Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4487	2024-12-17T21:35:36.138Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
4488	2024-12-17T21:35:36.138Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "48ded5c1-79a1-4769-bd57-fdb9f30214bf", "content-length": "2", "date": "Tue, 17 Dec 2024 21:35:35 GMT"} })
4489	2024-12-17T21:35:36.139Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4490	2024-12-17T21:35:36.139Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4491	2024-12-17T21:35:36.140Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4492	2024-12-17T21:35:36.141Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
4493	2024-12-17T21:35:36.141Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
4494	2024-12-17T21:35:36.141Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
4495	2024-12-17T21:35:36.141Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
4496	2024-12-17T21:35:36.142Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 207.64807ms iteration = 26 status = null
4497	2024-12-17T21:35:36.143Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 208.373079ms iteration = 26 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
4498	2024-12-17T21:35:36.143Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: d3278cd4-2587-4f0f-89dd-b4d4669a6ffe, lookup_type: ById(d3278cd4-2587-4f0f-89dd-b4d4669a6ffe) } result = Ok(())
4499	2024-12-17T21:35:36.143Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4500	2024-12-17T21:35:36.143Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4501	2024-12-17T21:35:36.145Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 210.490636ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
4502	2024-12-17T21:35:36.146Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 211.468933ms iteration = 26 status = 0
4503	2024-12-17T21:35:36.146Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
4504	2024-12-17T21:35:36.146Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
4505	2024-12-17T21:35:36.157Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 222.917243ms iteration = 26 status = {}
4506	2024-12-17T21:35:36.175Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 240.616442ms iteration = 26 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
4507	2024-12-17T21:35:36.179Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
4508	2024-12-17T21:35:36.179Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
4509	2024-12-17T21:35:36.179Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 244.747234ms iteration = 26 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
4510	2024-12-17T21:35:36.189Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc2599a7-07a1-4a06-b75a-3fa18e1c02f0", "content-length": "372", "date": "Tue, 17 Dec 2024 21:35:35 GMT"} })
4511	2024-12-17T21:35:36.189Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4512	2024-12-17T21:35:36.189Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4513	2024-12-17T21:35:36.210Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4514	2024-12-17T21:35:36.210Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4515	2024-12-17T21:35:36.211Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4516	2024-12-17T21:35:36.213Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
4517	2024-12-17T21:35:36.213Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4518	2024-12-17T21:35:36.213Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4519	2024-12-17T21:35:36.221Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4520	2024-12-17T21:35:36.222Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4521	2024-12-17T21:35:36.223Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 1a3e1a79-c78b-4fb4-921b-c5e1e32e8b40, rack_id: 8216c8d1-8011-434d-a57a-2b1ce2e89ad8, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(d6c2617a-6b89-4779-bc70-3002fb0a7e5e) } rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0
4522	2024-12-17T21:35:36.223Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4523	2024-12-17T21:35:36.223Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4524	2024-12-17T21:35:36.231Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
4525	2024-12-17T21:35:36.231Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
4526	2024-12-17T21:35:36.231Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "dc425fa7-678a-44a5-ad95-598af1d839d6", "date": "Tue, 17 Dec 2024 21:35:36 GMT"} })
4527	2024-12-17T21:35:36.231Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4528	2024-12-17T21:35:36.231Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4529	2024-12-17T21:35:36.232Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
4530	2024-12-17T21:35:36.232Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "88f6459d-8532-4319-9422-17064cc54a04", "content-length": "134", "date": "Tue, 17 Dec 2024 21:35:35 GMT"} })
4531	2024-12-17T21:35:36.232Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
4532	2024-12-17T21:35:36.232Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = d6c2617a-6b89-4779-bc70-3002fb0a7e5e
4533	2024-12-17T21:35:36.232Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
4534	2024-12-17T21:35:36.232Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
4535	2024-12-17T21:35:36.232Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4536	2024-12-17T21:35:36.232Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4537	2024-12-17T21:35:36.232Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4538	2024-12-17T21:35:36.232Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
4539	2024-12-17T21:35:36.233Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4540	2024-12-17T21:35:36.233Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
4541	2024-12-17T21:35:36.233Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
4542	2024-12-17T21:35:36.233Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4543	2024-12-17T21:35:36.233Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4544	2024-12-17T21:35:36.233Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4545	2024-12-17T21:35:36.233Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4546	2024-12-17T21:35:36.234Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
4547	2024-12-17T21:35:36.234Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "d3d3d165-8540-45a0-95fa-e9721182fbd7", "content-length": "2", "date": "Tue, 17 Dec 2024 21:35:35 GMT"} })
4548	2024-12-17T21:35:36.234Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
4549	2024-12-17T21:35:36.234Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "394f8f9a-304f-4ebe-9184-232660e70e9e", "content-length": "51", "date": "Tue, 17 Dec 2024 21:35:35 GMT"} })
4550	2024-12-17T21:35:36.234Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0
4551	2024-12-17T21:35:36.234Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
4552	2024-12-17T21:35:36.236Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4553	2024-12-17T21:35:36.236Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4554	2024-12-17T21:35:36.239Z	INFO	nexus (BackgroundTasks): VPC route manager sled d3278cd4-2587-4f0f-89dd-b4d4669a6ffe push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
4555	2024-12-17T21:35:36.239Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
4556	2024-12-17T21:35:36.241Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4557	2024-12-17T21:35:36.243Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
4558	2024-12-17T21:35:36.243Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4559	2024-12-17T21:35:36.243Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4560	2024-12-17T21:35:36.278Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
4561	2024-12-17T21:35:36.329Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "c05b888b-6ffc-4f23-8847-1d437652771b", "date": "Tue, 17 Dec 2024 21:35:36 GMT"} })
4562	2024-12-17T21:35:36.330Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch = Switch0
4563	2024-12-17T21:35:36.330Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
4564	2024-12-17T21:35:36.330Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b213c1e-d6c3-4cc3-b0bd-62ef8121593b", "content-length": "682", "date": "Tue, 17 Dec 2024 21:35:36 GMT"} })
4565	2024-12-17T21:35:36.330Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4566	2024-12-17T21:35:36.330Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4567	2024-12-17T21:35:36.330Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "642a60a8-2c89-4c37-8f3c-92628829d06f", "date": "Tue, 17 Dec 2024 21:35:35 GMT"} })
4568	2024-12-17T21:35:36.330Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "e53dcaf0-3fec-4f65-a6d0-a32be152818b", "content-length": "691", "date": "Tue, 17 Dec 2024 21:35:36 GMT"} })
4569	2024-12-17T21:35:36.330Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
4570	2024-12-17T21:35:36.330Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
4571	2024-12-17T21:35:36.330Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
4572	2024-12-17T21:35:36.332Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 397.482327ms iteration = 26 status = {}
4573	2024-12-17T21:35:36.332Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 397.738464ms iteration = 26 status = {}
4574	2024-12-17T21:35:36.333Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4575	2024-12-17T21:35:36.333Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4576	2024-12-17T21:35:36.337Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4577	2024-12-17T21:35:36.337Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4578	2024-12-17T21:35:36.338Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4579	2024-12-17T21:35:36.339Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4580	2024-12-17T21:35:36.346Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 411.859316ms iteration = 26 status = {}
4581	2024-12-17T21:35:40.393Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 1e7e902b-b0f6-405f-b44c-845e73d3565a roles = RoleSet { roles: {} } uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?sort_by=id_ascending
4582	2024-12-17T21:35:40.393Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 1e7e902b-b0f6-405f-b44c-845e73d3565a resource = Database result = Ok(()) uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?sort_by=id_ascending
4583	2024-12-17T21:35:40.396Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3615 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 1e7e902b-b0f6-405f-b44c-845e73d3565a response_code = 200 uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?sort_by=id_ascending
4584	2024-12-17T21:35:40.396Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 8e5aade8-461c-44e9-9a67-b2ef1721f7ff roles = RoleSet { roles: {} } uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMTkzZjYyMi00MGY1LTQyOGYtYmVhMy01MmNjNjdlNzhhMzkifX0%3D
4585	2024-12-17T21:35:40.397Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 8e5aade8-461c-44e9-9a67-b2ef1721f7ff resource = Database result = Ok(()) uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMTkzZjYyMi00MGY1LTQyOGYtYmVhMy01MmNjNjdlNzhhMzkifX0%3D
4586	2024-12-17T21:35:40.399Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2217 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50255 req_id = 8e5aade8-461c-44e9-9a67-b2ef1721f7ff response_code = 200 uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMTkzZjYyMi00MGY1LTQyOGYtYmVhMy01MmNjNjdlNzhhMzkifX0%3D
4587	2024-12-17T21:35:40.405Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 395 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = b49e0b05-de3e-4de5-9e07-b14b6e3e32d7 response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
4588	2024-12-17T21:35:45.934Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 77 reason = Timeout
4589	2024-12-17T21:35:45.934Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4590	2024-12-17T21:35:45.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4591	2024-12-17T21:35:45.934Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 77 reason = Timeout
4592	2024-12-17T21:35:45.934Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4593	2024-12-17T21:35:45.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4594	2024-12-17T21:35:45.936Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4595	2024-12-17T21:35:45.937Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4596	2024-12-17T21:35:45.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4597	2024-12-17T21:35:45.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4598	2024-12-17T21:35:45.939Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4599	2024-12-17T21:35:45.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4600	2024-12-17T21:35:45.939Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 5.326354ms iteration = 77 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
4601	2024-12-17T21:35:45.940Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4602	2024-12-17T21:35:45.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4603	2024-12-17T21:35:45.942Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4604	2024-12-17T21:35:45.944Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
4605	2024-12-17T21:35:45.944Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4606	2024-12-17T21:35:45.944Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4607	2024-12-17T21:35:45.955Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
4608	2024-12-17T21:35:45.955Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.483031ms iteration = 77 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
4609	2024-12-17T21:35:50.406Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 309 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = 9b6c7f68-154e-4cfa-8f2c-4d69df4efcfa response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
4610	2024-12-17T21:35:55.934Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 78 reason = Timeout
4611	2024-12-17T21:35:55.934Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4612	2024-12-17T21:35:55.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4613	2024-12-17T21:35:55.934Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 78 reason = Timeout
4614	2024-12-17T21:35:55.934Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4615	2024-12-17T21:35:55.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4616	2024-12-17T21:35:55.935Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4617	2024-12-17T21:35:55.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4618	2024-12-17T21:35:55.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4619	2024-12-17T21:35:55.938Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4620	2024-12-17T21:35:55.938Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4621	2024-12-17T21:35:55.938Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4622	2024-12-17T21:35:55.940Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4623	2024-12-17T21:35:55.940Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4624	2024-12-17T21:35:55.942Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4625	2024-12-17T21:35:55.945Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
4626	2024-12-17T21:35:55.945Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4627	2024-12-17T21:35:55.945Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4628	2024-12-17T21:35:55.955Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
4629	2024-12-17T21:35:55.955Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.584984ms iteration = 78 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
4630	2024-12-17T21:35:55.971Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 36.795976ms iteration = 78 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
4631	2024-12-17T21:36:00.406Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 403 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = ddbeb589-0529-4eb6-b164-35a9e19f488c response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
4632	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 14 reason = Timeout
4633	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 15 reason = Timeout
4634	2024-12-17T21:36:05.934Z	WARN	nexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5
4635	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 12.941\u{b5}s iteration = 15 status = {"enabled":false,"target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5"}
4636	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 27 reason = Timeout
4637	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 27 reason = Timeout
4638	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4639	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4640	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 16 reason = Timeout
4641	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 27 reason = Timeout
4642	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 14 reason = Timeout
4643	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 14 reason = Timeout
4644	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
4645	2024-12-17T21:36:05.934Z	INFO	nexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
4646	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4647	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4648	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
4649	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 29 reason = Timeout
4650	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 27 reason = Timeout
4651	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4652	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 27 reason = Timeout
4653	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4654	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 27 reason = Timeout
4655	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 27 reason = Timeout
4656	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 14 reason = Timeout
4657	2024-12-17T21:36:05.934Z	INFO	nexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-17 21:26:05.934396877 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
4658	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
4659	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
4660	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
4661	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 16 reason = Timeout
4662	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 14 reason = Timeout
4663	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 14 reason = Timeout
4664	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
4665	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4666	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = external_endpoints iteration = 14 reason = Timeout
4667	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4668	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 79 reason = Timeout
4669	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4670	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 27 reason = Timeout
4671	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
4672	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 27 reason = Timeout
4673	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 14 reason = Timeout
4674	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
4675	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
4676	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
4677	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 14 reason = Timeout
4678	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
4679	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 27 reason = Timeout
4680	2024-12-17T21:36:05.934Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
4681	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 79 reason = Timeout
4682	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4683	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4684	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 27 reason = Timeout
4685	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
4686	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 14 reason = Timeout
4687	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
4688	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 27 reason = Timeout
4689	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
4690	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 27 reason = Timeout
4691	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
4692	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 27 reason = Timeout
4693	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
4694	2024-12-17T21:36:05.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4695	2024-12-17T21:36:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4696	2024-12-17T21:36:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4697	2024-12-17T21:36:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4698	2024-12-17T21:36:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
4699	2024-12-17T21:36:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4700	2024-12-17T21:36:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
4701	2024-12-17T21:36:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4702	2024-12-17T21:36:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4703	2024-12-17T21:36:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
4704	2024-12-17T21:36:05.935Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("5dff2793-3ce8-4251-ba71-1a446966cbf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a761a47e-08b9-41c4-9bd8-8d77651dfa62.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a761a47e-08b9-41c4-9bd8-8d77651dfa62.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 2024, tv_nsec: 68774097 } })
4705	2024-12-17T21:36:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
4706	2024-12-17T21:36:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4707	2024-12-17T21:36:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4708	2024-12-17T21:36:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
4709	2024-12-17T21:36:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
4710	2024-12-17T21:36:05.935Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0005c310-624c-4312-b0fd-35f571746851.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6ac4d5e8-fcd9-4aa9-aed5-fb3dc293aac2.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9b804364-993d-47ba-81a1-d0ef32f96529.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("9b804364-993d-47ba-81a1-d0ef32f96529.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 2024, tv_nsec: 68866039 } })
4711	2024-12-17T21:36:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
4712	2024-12-17T21:36:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
4713	2024-12-17T21:36:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
4714	2024-12-17T21:36:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4715	2024-12-17T21:36:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4716	2024-12-17T21:36:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
4717	2024-12-17T21:36:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
4718	2024-12-17T21:36:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
4719	2024-12-17T21:36:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
4720	2024-12-17T21:36:05.935Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
4721	2024-12-17T21:36:05.935Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.004747ms iteration = 14 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
4722	2024-12-17T21:36:05.935Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
4723	2024-12-17T21:36:05.935Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.413688ms iteration = 14 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
4724	2024-12-17T21:36:05.935Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4725	2024-12-17T21:36:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4726	2024-12-17T21:36:05.937Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.461793ms iteration = 27 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
4727	2024-12-17T21:36:05.938Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4728	2024-12-17T21:36:05.938Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4729	2024-12-17T21:36:05.940Z	INFO	nexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-17 21:26:05.934396877 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
4730	2024-12-17T21:36:05.940Z	DEBG	nexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 5.992612ms iteration = 14 status = {"expiration":"2024-12-17T21:26:05.934396877Z","pruned":[]}
4731	2024-12-17T21:36:05.940Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
4732	2024-12-17T21:36:05.940Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4733	2024-12-17T21:36:05.940Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4734	2024-12-17T21:36:05.940Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
4735	2024-12-17T21:36:05.940Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4736	2024-12-17T21:36:05.940Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4737	2024-12-17T21:36:05.942Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4738	2024-12-17T21:36:05.942Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2024, tv_nsec: 76602009 } })
4739	2024-12-17T21:36:05.943Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
4740	2024-12-17T21:36:05.943Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4741	2024-12-17T21:36:05.943Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4742	2024-12-17T21:36:05.944Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2024, tv_nsec: 77738839 } })
4743	2024-12-17T21:36:05.944Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4744	2024-12-17T21:36:05.944Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
4745	2024-12-17T21:36:05.945Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
4746	2024-12-17T21:36:05.945Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4747	2024-12-17T21:36:05.946Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4748	2024-12-17T21:36:05.946Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
4749	2024-12-17T21:36:05.946Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4750	2024-12-17T21:36:05.946Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4751	2024-12-17T21:36:05.947Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4752	2024-12-17T21:36:05.948Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4753	2024-12-17T21:36:05.957Z	DEBG	nexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 23.167487ms iteration = 14 status = {"errors":[],"found_port_ok":[]}
4754	2024-12-17T21:36:05.957Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4755	2024-12-17T21:36:05.960Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
4756	2024-12-17T21:36:05.960Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
4757	2024-12-17T21:36:05.960Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
4758	2024-12-17T21:36:05.964Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2024, tv_nsec: 98056329 } })
4759	2024-12-17T21:36:05.964Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4760	2024-12-17T21:36:05.964Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
4761	2024-12-17T21:36:05.964Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4762	2024-12-17T21:36:05.964Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4763	2024-12-17T21:36:05.965Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4764	2024-12-17T21:36:05.966Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4765	2024-12-17T21:36:05.966Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4766	2024-12-17T21:36:05.969Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4767	2024-12-17T21:36:05.971Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
4768	2024-12-17T21:36:05.971Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4769	2024-12-17T21:36:05.971Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4770	2024-12-17T21:36:05.972Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
4771	2024-12-17T21:36:05.972Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 37.540556ms iteration = 27 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
4772	2024-12-17T21:36:05.975Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4773	2024-12-17T21:36:05.975Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4774	2024-12-17T21:36:05.988Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
4775	2024-12-17T21:36:05.989Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
4776	2024-12-17T21:36:05.991Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4777	2024-12-17T21:36:05.991Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4778	2024-12-17T21:36:05.993Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
4779	2024-12-17T21:36:05.993Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4780	2024-12-17T21:36:05.994Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4781	2024-12-17T21:36:05.994Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4782	2024-12-17T21:36:05.994Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4783	2024-12-17T21:36:05.994Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4784	2024-12-17T21:36:05.996Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4785	2024-12-17T21:36:05.997Z	DEBG	nexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
4786	2024-12-17T21:36:05.997Z	INFO	nexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
4787	2024-12-17T21:36:05.997Z	DEBG	nexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 62.685856ms iteration = 14 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
4788	2024-12-17T21:36:05.997Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
4789	2024-12-17T21:36:05.997Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
4790	2024-12-17T21:36:05.997Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
4791	2024-12-17T21:36:05.998Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
4792	2024-12-17T21:36:05.998Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4793	2024-12-17T21:36:05.999Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4794	2024-12-17T21:36:05.999Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
4795	2024-12-17T21:36:05.999Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4796	2024-12-17T21:36:05.999Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 65.318198ms iteration = 27 status = {"errors":[],"records_set_to_done":[]}
4797	2024-12-17T21:36:05.999Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4798	2024-12-17T21:36:06.001Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4799	2024-12-17T21:36:06.004Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
4800	2024-12-17T21:36:06.004Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
4801	2024-12-17T21:36:06.004Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
4802	2024-12-17T21:36:06.005Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4803	2024-12-17T21:36:06.008Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
4804	2024-12-17T21:36:06.008Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4805	2024-12-17T21:36:06.008Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
4806	2024-12-17T21:36:06.008Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
4807	2024-12-17T21:36:06.008Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
4808	2024-12-17T21:36:06.008Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
4809	2024-12-17T21:36:06.008Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
4810	2024-12-17T21:36:06.008Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4811	2024-12-17T21:36:06.008Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 73.820968ms iteration = 79 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
4812	2024-12-17T21:36:06.008Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4813	2024-12-17T21:36:06.009Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
4814	2024-12-17T21:36:06.009Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 75.154184ms iteration = 27 status = {"errors":[],"garbage_collect_requested":[]}
4815	2024-12-17T21:36:06.011Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
4816	2024-12-17T21:36:06.011Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4817	2024-12-17T21:36:06.011Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4818	2024-12-17T21:36:06.011Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4819	2024-12-17T21:36:06.012Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4820	2024-12-17T21:36:06.012Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4821	2024-12-17T21:36:06.013Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
4822	2024-12-17T21:36:06.013Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4823	2024-12-17T21:36:06.013Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
4824	2024-12-17T21:36:06.013Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
4825	2024-12-17T21:36:06.013Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4826	2024-12-17T21:36:06.013Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
4827	2024-12-17T21:36:06.016Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
4828	2024-12-17T21:36:06.017Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 83.300445ms iteration = 29 status = {"physical_disks_added":0}
4829	2024-12-17T21:36:06.017Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
4830	2024-12-17T21:36:06.017Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4831	2024-12-17T21:36:06.017Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
4832	2024-12-17T21:36:06.017Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4833	2024-12-17T21:36:06.017Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4834	2024-12-17T21:36:06.018Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4835	2024-12-17T21:36:06.020Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4836	2024-12-17T21:36:06.020Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 85.777561ms iteration = 27 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
4837	2024-12-17T21:36:06.020Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4838	2024-12-17T21:36:06.021Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4839	2024-12-17T21:36:06.025Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: d3278cd4-2587-4f0f-89dd-b4d4669a6ffe, lookup_type: ById(d3278cd4-2587-4f0f-89dd-b4d4669a6ffe) } result = Ok(())
4840	2024-12-17T21:36:06.025Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4841	2024-12-17T21:36:06.025Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4842	2024-12-17T21:36:06.027Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 92.832602ms iteration = 27 status = 0
4843	2024-12-17T21:36:06.029Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
4844	2024-12-17T21:36:06.029Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 95.472494ms iteration = 79 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
4845	2024-12-17T21:36:06.038Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
4846	2024-12-17T21:36:06.038Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
4847	2024-12-17T21:36:06.046Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
4848	2024-12-17T21:36:06.046Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
4849	2024-12-17T21:36:06.046Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
4850	2024-12-17T21:36:06.046Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4851	2024-12-17T21:36:06.046Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
4852	2024-12-17T21:36:06.046Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
4853	2024-12-17T21:36:06.046Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 112.258738ms iteration = 27 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
4854	2024-12-17T21:36:06.046Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4855	2024-12-17T21:36:06.048Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4856	2024-12-17T21:36:06.051Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
4857	2024-12-17T21:36:06.051Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
4858	2024-12-17T21:36:06.051Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
4859	2024-12-17T21:36:06.051Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4860	2024-12-17T21:36:06.051Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 116.826601ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
4861	2024-12-17T21:36:06.051Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4862	2024-12-17T21:36:06.052Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
4863	2024-12-17T21:36:06.052Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4864	2024-12-17T21:36:06.052Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4865	2024-12-17T21:36:06.053Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4866	2024-12-17T21:36:06.055Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 120.82204ms iteration = 27 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
4867	2024-12-17T21:36:06.055Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
4868	2024-12-17T21:36:06.055Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4869	2024-12-17T21:36:06.056Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4870	2024-12-17T21:36:06.056Z	DEBG	nexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
4871	2024-12-17T21:36:06.056Z	INFO	nexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
4872	2024-12-17T21:36:06.056Z	DEBG	nexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 122.040373ms iteration = 14 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
4873	2024-12-17T21:36:06.062Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4874	2024-12-17T21:36:06.062Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4875	2024-12-17T21:36:06.063Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4876	2024-12-17T21:36:06.064Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4877	2024-12-17T21:36:06.067Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
4878	2024-12-17T21:36:06.067Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4879	2024-12-17T21:36:06.067Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
4880	2024-12-17T21:36:06.067Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4881	2024-12-17T21:36:06.067Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4882	2024-12-17T21:36:06.067Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4883	2024-12-17T21:36:06.089Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
4884	2024-12-17T21:36:06.089Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = ea6acdde-77d4-4bf6-b578-d9a0a29ac37b dns_zone_name = oxide.test found_so_far = 0 version = 2
4885	2024-12-17T21:36:06.089Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4886	2024-12-17T21:36:06.089Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4887	2024-12-17T21:36:06.092Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
4888	2024-12-17T21:36:06.092Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 574f0c35-b8f2-48fe-831c-78afc4a62aaa dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
4889	2024-12-17T21:36:06.092Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4890	2024-12-17T21:36:06.092Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4891	2024-12-17T21:36:06.093Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
4892	2024-12-17T21:36:06.115Z	DEBG	nexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-17T21:23:03.821125Z, time_modified: 2024-12-17T21:23:03.821125Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 81e78e1e-e55a-44f9-abdc-c9c916e1d9fd, db_silo: Silo { identity: SiloIdentity { id: 81e78e1e-e55a-44f9-abdc-c9c916e1d9fd, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-17T21:23:49.443930Z, time_modified: 2024-12-17T21:23:49.443930Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "e275643b21ab02733b9616c9cd3b0193e27c7f6294b7d310d6f1726c72768e50" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 81e78e1e-e55a-44f9-abdc-c9c916e1d9fd, db_silo: Silo { identity: SiloIdentity { id: 81e78e1e-e55a-44f9-abdc-c9c916e1d9fd, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-17T21:23:49.443930Z, time_modified: 2024-12-17T21:23:49.443930Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "e275643b21ab02733b9616c9cd3b0193e27c7f6294b7d310d6f1726c72768e50" }] }) }
4893	2024-12-17T21:36:06.115Z	DEBG	nexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 181.450281ms iteration = 14 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"81e78e1e-e55a-44f9-abdc-c9c916e1d9fd","tls_certs":["e275643b21ab02733b9616c9cd3b0193e27c7f6294b7d310d6f1726c72768e50"]}},"default_endpoint":{"silo_id":"81e78e1e-e55a-44f9-abdc-c9c916e1d9fd","tls_certs":["e275643b21ab02733b9616c9cd3b0193e27c7f6294b7d310d6f1726c72768e50"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
4894	2024-12-17T21:36:06.132Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4895	2024-12-17T21:36:06.135Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
4896	2024-12-17T21:36:06.135Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4897	2024-12-17T21:36:06.135Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4898	2024-12-17T21:36:06.147Z	WARN	nexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
4899	2024-12-17T21:36:06.147Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 212.543922ms iteration = 14 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
4900	2024-12-17T21:36:06.252Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
4901	2024-12-17T21:36:06.252Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4902	2024-12-17T21:36:06.252Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f27e3c8-85b7-4fba-b6f7-c9f7bc228154", "content-length": "26", "date": "Tue, 17 Dec 2024 21:36:05 GMT"} })
4903	2024-12-17T21:36:06.252Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4904	2024-12-17T21:36:06.253Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
4905	2024-12-17T21:36:06.254Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
4906	2024-12-17T21:36:06.254Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4907	2024-12-17T21:36:06.254Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "68a93e02-146c-4c5b-845c-ad9e244e6135", "content-length": "26", "date": "Tue, 17 Dec 2024 21:36:05 GMT"} })
4908	2024-12-17T21:36:06.254Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4909	2024-12-17T21:36:06.256Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
4910	2024-12-17T21:36:06.256Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
4911	2024-12-17T21:36:06.279Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 zone_address = fd00:1122:3344:101::2
4912	2024-12-17T21:36:06.279Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4913	2024-12-17T21:36:06.279Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e498069-5ea4-40e5-9b0a-afc613ac2309", "content-length": "26", "date": "Tue, 17 Dec 2024 21:36:05 GMT"} })
4914	2024-12-17T21:36:06.279Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4915	2024-12-17T21:36:06.281Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
4916	2024-12-17T21:36:06.284Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
4917	2024-12-17T21:36:06.286Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4918	2024-12-17T21:36:06.286Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = ea6acdde-77d4-4bf6-b578-d9a0a29ac37b dns_zone_name = oxide.test found_so_far = 1 version = 2
4919	2024-12-17T21:36:06.286Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
4920	2024-12-17T21:36:06.286Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-17 21:23:49.491999 UTC dns_group = external generation = 2
4921	2024-12-17T21:36:06.286Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 352.384097ms iteration = 14 status = {"generation":2}
4922	2024-12-17T21:36:06.289Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
4923	2024-12-17T21:36:06.289Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4924	2024-12-17T21:36:06.289Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4925	2024-12-17T21:36:06.290Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
4926	2024-12-17T21:36:06.346Z	INFO	nexus (BackgroundTasks): VPC route manager sled d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
4927	2024-12-17T21:36:06.346Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
4928	2024-12-17T21:36:06.348Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 574f0c35-b8f2-48fe-831c-78afc4a62aaa dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
4929	2024-12-17T21:36:06.348Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
4930	2024-12-17T21:36:06.348Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-17 21:23:48.446255 UTC dns_group = internal generation = 1
4931	2024-12-17T21:36:06.348Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 413.978754ms iteration = 14 status = {"generation":1}
4932	2024-12-17T21:36:06.369Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "11133436-3016-428c-98e4-3749d186ef80", "content-length": "372", "date": "Tue, 17 Dec 2024 21:36:06 GMT"} })
4933	2024-12-17T21:36:06.369Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4934	2024-12-17T21:36:06.369Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "522f5b2e-b207-4af3-bd81-05ceaa254a91", "content-length": "691", "date": "Tue, 17 Dec 2024 21:36:06 GMT"} })
4935	2024-12-17T21:36:06.369Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
4936	2024-12-17T21:36:06.369Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
4937	2024-12-17T21:36:06.369Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
4938	2024-12-17T21:36:06.369Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4939	2024-12-17T21:36:06.371Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 436.817572ms iteration = 27 status = {}
4940	2024-12-17T21:36:06.411Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
4941	2024-12-17T21:36:06.411Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
4942	2024-12-17T21:36:06.411Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "e8a8fca3-e6cf-43b4-986a-9454bc30d730", "date": "Tue, 17 Dec 2024 21:36:06 GMT"} })
4943	2024-12-17T21:36:06.411Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4944	2024-12-17T21:36:06.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4945	2024-12-17T21:36:06.414Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4946	2024-12-17T21:36:06.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4947	2024-12-17T21:36:06.417Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4948	2024-12-17T21:36:06.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4949	2024-12-17T21:36:06.421Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 487.156134ms iteration = 27 status = {}
4950	2024-12-17T21:36:06.422Z	INFO	nexus (BackgroundTasks): VPC route manager sled d3278cd4-2587-4f0f-89dd-b4d4669a6ffe push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
4951	2024-12-17T21:36:06.422Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
4952	2024-12-17T21:36:06.422Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "b762361b-09d7-41eb-940e-0038cd101cb5", "date": "Tue, 17 Dec 2024 21:36:06 GMT"} })
4953	2024-12-17T21:36:06.423Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
4954	2024-12-17T21:36:06.423Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "60f7acd8-bb99-42f0-8de2-0ab08839cc90", "content-length": "2", "date": "Tue, 17 Dec 2024 21:36:05 GMT"} })
4955	2024-12-17T21:36:06.424Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 490.407272ms iteration = 27 status = {}
4956	2024-12-17T21:36:06.425Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4957	2024-12-17T21:36:06.425Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4958	2024-12-17T21:36:06.427Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 493.28212ms iteration = 27 status = null
4959	2024-12-17T21:36:06.444Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
4960	2024-12-17T21:36:06.444Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "e1aef258-ed5c-4ea4-b9fa-6fb41ff2996c", "date": "Tue, 17 Dec 2024 21:36:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4961	2024-12-17T21:36:06.444Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4962	2024-12-17T21:36:06.446Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "5fbfac43-866f-4689-b43e-cded4ce3a8bf", "date": "Tue, 17 Dec 2024 21:36:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4963	2024-12-17T21:36:06.446Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4964	2024-12-17T21:36:06.448Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "d8b816b6-74d4-4fd4-91d8-f20af619f9f5", "date": "Tue, 17 Dec 2024 21:36:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4965	2024-12-17T21:36:06.448Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4966	2024-12-17T21:36:06.450Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4967	2024-12-17T21:36:06.450Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 516.039465ms iteration = 16 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
4968	2024-12-17T21:36:06.484Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
4969	2024-12-17T21:36:06.484Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "d04d294b-0e0d-441a-b753-d7f07a871ebf", "date": "Tue, 17 Dec 2024 21:36:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4970	2024-12-17T21:36:06.484Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4971	2024-12-17T21:36:06.486Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "b96ef450-64b1-4575-b446-731b336e13da", "date": "Tue, 17 Dec 2024 21:36:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4972	2024-12-17T21:36:06.486Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4973	2024-12-17T21:36:06.488Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4974	2024-12-17T21:36:06.488Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 554.441635ms iteration = 16 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
4975	2024-12-17T21:36:06.549Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4976	2024-12-17T21:36:06.549Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4977	2024-12-17T21:36:06.551Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 1a3e1a79-c78b-4fb4-921b-c5e1e32e8b40, rack_id: 8216c8d1-8011-434d-a57a-2b1ce2e89ad8, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(d6c2617a-6b89-4779-bc70-3002fb0a7e5e) } rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0
4978	2024-12-17T21:36:06.551Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4979	2024-12-17T21:36:06.551Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4980	2024-12-17T21:36:06.561Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
4981	2024-12-17T21:36:06.562Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "d351e5b7-5e80-42f8-8f74-83cb6a39363f", "content-length": "134", "date": "Tue, 17 Dec 2024 21:36:05 GMT"} })
4982	2024-12-17T21:36:06.562Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
4983	2024-12-17T21:36:06.562Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = d6c2617a-6b89-4779-bc70-3002fb0a7e5e
4984	2024-12-17T21:36:06.562Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
4985	2024-12-17T21:36:06.562Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
4986	2024-12-17T21:36:06.562Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4987	2024-12-17T21:36:06.562Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4988	2024-12-17T21:36:06.562Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4989	2024-12-17T21:36:06.562Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
4990	2024-12-17T21:36:06.562Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4991	2024-12-17T21:36:06.562Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
4992	2024-12-17T21:36:06.562Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
4993	2024-12-17T21:36:06.562Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4994	2024-12-17T21:36:06.563Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4995	2024-12-17T21:36:06.565Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
4996	2024-12-17T21:36:06.565Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "be569fa2-53eb-413c-81a4-b9f7113d500b", "content-length": "2", "date": "Tue, 17 Dec 2024 21:36:05 GMT"} })
4997	2024-12-17T21:36:06.565Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
4998	2024-12-17T21:36:06.565Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "38d5f587-5852-4701-af87-8f072c57eff8", "content-length": "51", "date": "Tue, 17 Dec 2024 21:36:05 GMT"} })
4999	2024-12-17T21:36:06.565Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0
5000	2024-12-17T21:36:06.565Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
5001	2024-12-17T21:36:06.653Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "cbddc3c0-3473-42e3-ad1e-1d45f59d9d3c", "date": "Tue, 17 Dec 2024 21:36:06 GMT"} })
5002	2024-12-17T21:36:06.653Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch = Switch0
5003	2024-12-17T21:36:06.653Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
5004	2024-12-17T21:36:06.653Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e20fab8-5e65-47e4-96b9-5e32d73c7a43", "content-length": "682", "date": "Tue, 17 Dec 2024 21:36:06 GMT"} })
5005	2024-12-17T21:36:06.653Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5006	2024-12-17T21:36:06.654Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5007	2024-12-17T21:36:06.657Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5008	2024-12-17T21:36:06.657Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5009	2024-12-17T21:36:06.660Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5010	2024-12-17T21:36:06.661Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5011	2024-12-17T21:36:06.663Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5012	2024-12-17T21:36:06.663Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5013	2024-12-17T21:36:06.670Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 735.862845ms iteration = 27 status = {}
5014	2024-12-17T21:36:10.405Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 450 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = e49740b6-6d93-4a5c-82e9-fa40bb60f991 response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
5015	2024-12-17T21:36:15.933Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 80 reason = Timeout
5016	2024-12-17T21:36:15.933Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5017	2024-12-17T21:36:15.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5018	2024-12-17T21:36:15.934Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 80 reason = Timeout
5019	2024-12-17T21:36:15.934Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5020	2024-12-17T21:36:15.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5021	2024-12-17T21:36:15.935Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5022	2024-12-17T21:36:15.937Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5023	2024-12-17T21:36:15.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5024	2024-12-17T21:36:15.938Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5025	2024-12-17T21:36:15.938Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5026	2024-12-17T21:36:15.938Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5027	2024-12-17T21:36:15.940Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5028	2024-12-17T21:36:15.940Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5029	2024-12-17T21:36:15.942Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5030	2024-12-17T21:36:15.944Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
5031	2024-12-17T21:36:15.944Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5032	2024-12-17T21:36:15.944Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5033	2024-12-17T21:36:15.956Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
5034	2024-12-17T21:36:15.956Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.633342ms iteration = 80 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
5035	2024-12-17T21:36:15.967Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 33.202849ms iteration = 80 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
5036	2024-12-17T21:36:20.405Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 521 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = 98bd95d0-17c3-49d4-a734-e07212a0aea2 response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
5037	2024-12-17T21:36:25.933Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 81 reason = Timeout
5038	2024-12-17T21:36:25.933Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5039	2024-12-17T21:36:25.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5040	2024-12-17T21:36:25.934Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 81 reason = Timeout
5041	2024-12-17T21:36:25.934Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5042	2024-12-17T21:36:25.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5043	2024-12-17T21:36:25.936Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5044	2024-12-17T21:36:25.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5045	2024-12-17T21:36:25.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5046	2024-12-17T21:36:25.938Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5047	2024-12-17T21:36:25.938Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5048	2024-12-17T21:36:25.938Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.222845ms iteration = 81 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
5049	2024-12-17T21:36:25.938Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5050	2024-12-17T21:36:25.940Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5051	2024-12-17T21:36:25.940Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5052	2024-12-17T21:36:25.941Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5053	2024-12-17T21:36:25.943Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
5054	2024-12-17T21:36:25.943Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5055	2024-12-17T21:36:25.943Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5056	2024-12-17T21:36:25.955Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
5057	2024-12-17T21:36:25.955Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.659086ms iteration = 81 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
5058	2024-12-17T21:36:30.404Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 294 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = e7ebc11a-306b-4f5b-ba33-2333bfd6007a response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
5059	2024-12-17T21:36:35.933Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 82 reason = Timeout
5060	2024-12-17T21:36:35.933Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 28 reason = Timeout
5061	2024-12-17T21:36:35.933Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5062	2024-12-17T21:36:35.933Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5063	2024-12-17T21:36:35.933Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 28 reason = Timeout
5064	2024-12-17T21:36:35.933Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 28 reason = Timeout
5065	2024-12-17T21:36:35.933Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
5066	2024-12-17T21:36:35.933Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5067	2024-12-17T21:36:35.933Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 28 reason = Timeout
5068	2024-12-17T21:36:35.933Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 28 reason = Timeout
5069	2024-12-17T21:36:35.933Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5070	2024-12-17T21:36:35.933Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5071	2024-12-17T21:36:35.933Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 28 reason = Timeout
5072	2024-12-17T21:36:35.933Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 28 reason = Timeout
5073	2024-12-17T21:36:35.933Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
5074	2024-12-17T21:36:35.933Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 30 reason = Timeout
5075	2024-12-17T21:36:35.933Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5076	2024-12-17T21:36:35.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5077	2024-12-17T21:36:35.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5078	2024-12-17T21:36:35.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5079	2024-12-17T21:36:35.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5080	2024-12-17T21:36:35.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
5081	2024-12-17T21:36:35.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5082	2024-12-17T21:36:35.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
5083	2024-12-17T21:36:35.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5084	2024-12-17T21:36:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 28 reason = Timeout
5085	2024-12-17T21:36:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 28 reason = Timeout
5086	2024-12-17T21:36:35.934Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
5087	2024-12-17T21:36:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5088	2024-12-17T21:36:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5089	2024-12-17T21:36:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 28 reason = Timeout
5090	2024-12-17T21:36:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5091	2024-12-17T21:36:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 28 reason = Timeout
5092	2024-12-17T21:36:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
5093	2024-12-17T21:36:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 28 reason = Timeout
5094	2024-12-17T21:36:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
5095	2024-12-17T21:36:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 28 reason = Timeout
5096	2024-12-17T21:36:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
5097	2024-12-17T21:36:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 82 reason = Timeout
5098	2024-12-17T21:36:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5099	2024-12-17T21:36:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 28 reason = Timeout
5100	2024-12-17T21:36:35.934Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
5101	2024-12-17T21:36:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5102	2024-12-17T21:36:35.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5103	2024-12-17T21:36:35.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
5104	2024-12-17T21:36:35.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5105	2024-12-17T21:36:35.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
5106	2024-12-17T21:36:35.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5107	2024-12-17T21:36:35.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
5108	2024-12-17T21:36:35.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
5109	2024-12-17T21:36:35.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
5110	2024-12-17T21:36:35.935Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5111	2024-12-17T21:36:35.937Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2054, tv_nsec: 71125141 } })
5112	2024-12-17T21:36:35.937Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5113	2024-12-17T21:36:35.937Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
5114	2024-12-17T21:36:35.937Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.979858ms iteration = 28 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
5115	2024-12-17T21:36:35.938Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5116	2024-12-17T21:36:35.938Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5117	2024-12-17T21:36:35.938Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5118	2024-12-17T21:36:35.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
5119	2024-12-17T21:36:35.939Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5120	2024-12-17T21:36:35.940Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5121	2024-12-17T21:36:35.940Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5122	2024-12-17T21:36:35.940Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5123	2024-12-17T21:36:35.941Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2054, tv_nsec: 75422037 } })
5124	2024-12-17T21:36:35.941Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
5125	2024-12-17T21:36:35.942Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
5126	2024-12-17T21:36:35.942Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5127	2024-12-17T21:36:35.942Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5128	2024-12-17T21:36:35.943Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
5129	2024-12-17T21:36:35.943Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5130	2024-12-17T21:36:35.943Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5131	2024-12-17T21:36:35.943Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5132	2024-12-17T21:36:35.943Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5133	2024-12-17T21:36:35.946Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5134	2024-12-17T21:36:35.948Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5135	2024-12-17T21:36:35.948Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5136	2024-12-17T21:36:35.948Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5137	2024-12-17T21:36:35.948Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5138	2024-12-17T21:36:35.949Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
5139	2024-12-17T21:36:35.949Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5140	2024-12-17T21:36:35.949Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
5141	2024-12-17T21:36:35.951Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5142	2024-12-17T21:36:35.951Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5143	2024-12-17T21:36:35.952Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2054, tv_nsec: 86249810 } })
5144	2024-12-17T21:36:35.952Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
5145	2024-12-17T21:36:35.953Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5146	2024-12-17T21:36:35.956Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
5147	2024-12-17T21:36:35.956Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5148	2024-12-17T21:36:35.956Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5149	2024-12-17T21:36:35.984Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
5150	2024-12-17T21:36:35.985Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
5151	2024-12-17T21:36:35.989Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5152	2024-12-17T21:36:35.990Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 55.595525ms iteration = 28 status = {"errors":[],"garbage_collect_requested":[]}
5153	2024-12-17T21:36:35.990Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5154	2024-12-17T21:36:35.990Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5155	2024-12-17T21:36:35.992Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
5156	2024-12-17T21:36:35.992Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5157	2024-12-17T21:36:35.992Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5158	2024-12-17T21:36:35.993Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5159	2024-12-17T21:36:35.993Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5160	2024-12-17T21:36:35.993Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 59.255363ms iteration = 28 status = {"errors":[],"records_set_to_done":[]}
5161	2024-12-17T21:36:35.994Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5162	2024-12-17T21:36:35.995Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5163	2024-12-17T21:36:35.995Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5164	2024-12-17T21:36:35.995Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5165	2024-12-17T21:36:35.996Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5166	2024-12-17T21:36:35.996Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5167	2024-12-17T21:36:35.997Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 62.515002ms iteration = 82 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
5168	2024-12-17T21:36:35.998Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
5169	2024-12-17T21:36:35.998Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5170	2024-12-17T21:36:35.998Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5171	2024-12-17T21:36:35.998Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
5172	2024-12-17T21:36:35.998Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5173	2024-12-17T21:36:35.998Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5174	2024-12-17T21:36:36.000Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
5175	2024-12-17T21:36:36.000Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
5176	2024-12-17T21:36:36.001Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
5177	2024-12-17T21:36:36.001Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
5178	2024-12-17T21:36:36.001Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5179	2024-12-17T21:36:36.002Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
5180	2024-12-17T21:36:36.004Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
5181	2024-12-17T21:36:36.004Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
5182	2024-12-17T21:36:36.004Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5183	2024-12-17T21:36:36.004Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
5184	2024-12-17T21:36:36.004Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
5185	2024-12-17T21:36:36.004Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 70.325813ms iteration = 28 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
5186	2024-12-17T21:36:36.008Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
5187	2024-12-17T21:36:36.008Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 75.502864ms iteration = 82 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
5188	2024-12-17T21:36:36.012Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 79.211794ms iteration = 30 status = {"physical_disks_added":0}
5189	2024-12-17T21:36:36.020Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 87.158419ms iteration = 28 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
5190	2024-12-17T21:36:36.022Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
5191	2024-12-17T21:36:36.022Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5192	2024-12-17T21:36:36.023Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "07abc218-a211-44d2-8966-1cef8cfba946", "content-length": "26", "date": "Tue, 17 Dec 2024 21:36:35 GMT"} })
5193	2024-12-17T21:36:36.023Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5194	2024-12-17T21:36:36.024Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
5195	2024-12-17T21:36:36.024Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5196	2024-12-17T21:36:36.025Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5197	2024-12-17T21:36:36.025Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5198	2024-12-17T21:36:36.027Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: d3278cd4-2587-4f0f-89dd-b4d4669a6ffe, lookup_type: ById(d3278cd4-2587-4f0f-89dd-b4d4669a6ffe) } result = Ok(())
5199	2024-12-17T21:36:36.027Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5200	2024-12-17T21:36:36.027Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5201	2024-12-17T21:36:36.030Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 96.637966ms iteration = 28 status = 0
5202	2024-12-17T21:36:36.030Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
5203	2024-12-17T21:36:36.030Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5204	2024-12-17T21:36:36.031Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a9020f0e-96f2-4461-af61-c44389fc27be", "content-length": "26", "date": "Tue, 17 Dec 2024 21:36:35 GMT"} })
5205	2024-12-17T21:36:36.031Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5206	2024-12-17T21:36:36.032Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 97.726754ms iteration = 28 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
5207	2024-12-17T21:36:36.033Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
5208	2024-12-17T21:36:36.039Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
5209	2024-12-17T21:36:36.039Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
5210	2024-12-17T21:36:36.039Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 104.534239ms iteration = 28 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
5211	2024-12-17T21:36:36.048Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 zone_address = fd00:1122:3344:101::2
5212	2024-12-17T21:36:36.048Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5213	2024-12-17T21:36:36.048Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "42903c82-7288-4eb8-9c79-23722eaa2f39", "content-length": "26", "date": "Tue, 17 Dec 2024 21:36:35 GMT"} })
5214	2024-12-17T21:36:36.048Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5215	2024-12-17T21:36:36.050Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
5216	2024-12-17T21:36:36.065Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 130.529063ms iteration = 28 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
5217	2024-12-17T21:36:36.089Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
5218	2024-12-17T21:36:36.090Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec547b13-ec4e-42a1-bff9-8b10615467b1", "content-length": "2", "date": "Tue, 17 Dec 2024 21:36:35 GMT"} })
5219	2024-12-17T21:36:36.092Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5220	2024-12-17T21:36:36.093Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5221	2024-12-17T21:36:36.098Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5222	2024-12-17T21:36:36.098Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5223	2024-12-17T21:36:36.107Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 174.076541ms iteration = 28 status = {}
5224	2024-12-17T21:36:36.140Z	INFO	nexus (BackgroundTasks): VPC route manager sled d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
5225	2024-12-17T21:36:36.140Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
5226	2024-12-17T21:36:36.142Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 208.752689ms iteration = 28 status = null
5227	2024-12-17T21:36:36.142Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5228	2024-12-17T21:36:36.144Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
5229	2024-12-17T21:36:36.144Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5230	2024-12-17T21:36:36.144Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5231	2024-12-17T21:36:36.145Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5232	2024-12-17T21:36:36.147Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
5233	2024-12-17T21:36:36.147Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5234	2024-12-17T21:36:36.148Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5235	2024-12-17T21:36:36.177Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "75b08e20-4fe1-47ab-9959-a5cedaef93d9", "content-length": "372", "date": "Tue, 17 Dec 2024 21:36:36 GMT"} })
5236	2024-12-17T21:36:36.177Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5237	2024-12-17T21:36:36.177Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5238	2024-12-17T21:36:36.198Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5239	2024-12-17T21:36:36.199Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5240	2024-12-17T21:36:36.200Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 1a3e1a79-c78b-4fb4-921b-c5e1e32e8b40, rack_id: 8216c8d1-8011-434d-a57a-2b1ce2e89ad8, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(d6c2617a-6b89-4779-bc70-3002fb0a7e5e) } rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0
5241	2024-12-17T21:36:36.200Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5242	2024-12-17T21:36:36.200Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5243	2024-12-17T21:36:36.208Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
5244	2024-12-17T21:36:36.208Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "4deef23c-6ac8-443f-b822-1e8b3b533fc0", "content-length": "134", "date": "Tue, 17 Dec 2024 21:36:35 GMT"} })
5245	2024-12-17T21:36:36.208Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
5246	2024-12-17T21:36:36.208Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = d6c2617a-6b89-4779-bc70-3002fb0a7e5e
5247	2024-12-17T21:36:36.209Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
5248	2024-12-17T21:36:36.209Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
5249	2024-12-17T21:36:36.209Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5250	2024-12-17T21:36:36.209Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5251	2024-12-17T21:36:36.209Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5252	2024-12-17T21:36:36.209Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
5253	2024-12-17T21:36:36.209Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5254	2024-12-17T21:36:36.209Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
5255	2024-12-17T21:36:36.209Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
5256	2024-12-17T21:36:36.209Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5257	2024-12-17T21:36:36.209Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5258	2024-12-17T21:36:36.211Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
5259	2024-12-17T21:36:36.211Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "dce76a65-c4af-43b8-81ed-83f35311caac", "content-length": "2", "date": "Tue, 17 Dec 2024 21:36:35 GMT"} })
5260	2024-12-17T21:36:36.211Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
5261	2024-12-17T21:36:36.211Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0286e37-7e84-482e-af18-79591302e597", "content-length": "51", "date": "Tue, 17 Dec 2024 21:36:35 GMT"} })
5262	2024-12-17T21:36:36.211Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0
5263	2024-12-17T21:36:36.211Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
5264	2024-12-17T21:36:36.218Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
5265	2024-12-17T21:36:36.218Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
5266	2024-12-17T21:36:36.235Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
5267	2024-12-17T21:36:36.308Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "38eb30e2-64ca-4a33-b6b8-02c39050420b", "date": "Tue, 17 Dec 2024 21:36:36 GMT"} })
5268	2024-12-17T21:36:36.308Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch = Switch0
5269	2024-12-17T21:36:36.308Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
5270	2024-12-17T21:36:36.308Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "96b15cda-f250-4186-9472-f8baf41e7789", "date": "Tue, 17 Dec 2024 21:36:36 GMT"} })
5271	2024-12-17T21:36:36.308Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5272	2024-12-17T21:36:36.309Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5273	2024-12-17T21:36:36.309Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "d8c0aafb-c9ff-4a76-b145-7ed7d3eabd35", "content-length": "691", "date": "Tue, 17 Dec 2024 21:36:35 GMT"} })
5274	2024-12-17T21:36:36.309Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
5275	2024-12-17T21:36:36.309Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "082410b3-fa17-425e-9b5f-fc25892cdd6a", "content-length": "682", "date": "Tue, 17 Dec 2024 21:36:36 GMT"} })
5276	2024-12-17T21:36:36.309Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
5277	2024-12-17T21:36:36.309Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
5278	2024-12-17T21:36:36.309Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5279	2024-12-17T21:36:36.309Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5280	2024-12-17T21:36:36.310Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 376.677814ms iteration = 28 status = {}
5281	2024-12-17T21:36:36.311Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5282	2024-12-17T21:36:36.311Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5283	2024-12-17T21:36:36.312Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5284	2024-12-17T21:36:36.313Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5285	2024-12-17T21:36:36.314Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5286	2024-12-17T21:36:36.314Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5287	2024-12-17T21:36:36.316Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5288	2024-12-17T21:36:36.316Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5289	2024-12-17T21:36:36.318Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5290	2024-12-17T21:36:36.318Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5291	2024-12-17T21:36:36.318Z	INFO	nexus (BackgroundTasks): VPC route manager sled d3278cd4-2587-4f0f-89dd-b4d4669a6ffe push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
5292	2024-12-17T21:36:36.318Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
5293	2024-12-17T21:36:36.318Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "1d5b9bb7-8fe2-4265-9f77-16e9ccbd21a5", "date": "Tue, 17 Dec 2024 21:36:36 GMT"} })
5294	2024-12-17T21:36:36.320Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 385.75328ms iteration = 28 status = {}
5295	2024-12-17T21:36:36.324Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 391.369432ms iteration = 28 status = {}
5296	2024-12-17T21:36:36.702Z	DEBG	nexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
5297	2024-12-17T21:36:36.702Z	DEBG	nexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
5298	2024-12-17T21:36:36.733Z	DEBG	nexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
5299	2024-12-17T21:36:36.733Z	INFO	nexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:52079
5300	2024-12-17T21:36:36.733Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:52079 req_id = e1e19fe4-dae8-4287-a45a-4a8059f68a3b roles = RoleSet { roles: {} } uri = /metrics/producers
5301	2024-12-17T21:36:36.733Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:52079 req_id = e1e19fe4-dae8-4287-a45a-4a8059f68a3b resource = Database result = Ok(()) uri = /metrics/producers
5302	2024-12-17T21:36:36.767Z	INFO	nexus: assigned collector to new producer collector_id = 952238fd-7df7-4f2b-b8ee-8be2d7488464 file = nexus/src/app/oximeter.rs:76 producer_id = 2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
5303	2024-12-17T21:36:36.767Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 34244 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:52079 req_id = e1e19fe4-dae8-4287-a45a-4a8059f68a3b response_code = 201 uri = /metrics/producers
5304	2024-12-17T21:36:36.767Z	DEBG	nexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "e1e19fe4-dae8-4287-a45a-4a8059f68a3b", "content-length": "41", "date": "Tue, 17 Dec 2024 21:36:36 GMT"} })
5305	2024-12-17T21:36:36.769Z	DEBG	nexus (producer-registration-task): registered with nexus successfully
5306	2024-12-17T21:36:36.769Z	DEBG	nexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
5307	2024-12-17T21:36:40.404Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 425 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = c1e9f255-6d40-44d5-a389-16d11391900f response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
5308	2024-12-17T21:36:45.933Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 83 reason = Timeout
5309	2024-12-17T21:36:45.933Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5310	2024-12-17T21:36:45.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5311	2024-12-17T21:36:45.934Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 83 reason = Timeout
5312	2024-12-17T21:36:45.934Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5313	2024-12-17T21:36:45.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5314	2024-12-17T21:36:45.936Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5315	2024-12-17T21:36:45.938Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5316	2024-12-17T21:36:45.938Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5317	2024-12-17T21:36:45.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5318	2024-12-17T21:36:45.941Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5319	2024-12-17T21:36:45.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5320	2024-12-17T21:36:45.943Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5321	2024-12-17T21:36:45.945Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
5322	2024-12-17T21:36:45.945Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5323	2024-12-17T21:36:45.945Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5324	2024-12-17T21:36:45.957Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
5325	2024-12-17T21:36:45.957Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.263009ms iteration = 83 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
5326	2024-12-17T21:36:45.962Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5327	2024-12-17T21:36:45.962Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5328	2024-12-17T21:36:45.990Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 55.828601ms iteration = 83 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
5329	2024-12-17T21:36:50.405Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 310 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = 7a2b5034-0622-4481-8c15-4c0b179827da response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
5330	2024-12-17T21:36:55.933Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 84 reason = Timeout
5331	2024-12-17T21:36:55.933Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5332	2024-12-17T21:36:55.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5333	2024-12-17T21:36:55.934Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 84 reason = Timeout
5334	2024-12-17T21:36:55.934Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5335	2024-12-17T21:36:55.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5336	2024-12-17T21:36:55.935Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5337	2024-12-17T21:36:55.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5338	2024-12-17T21:36:55.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5339	2024-12-17T21:36:55.938Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.847884ms iteration = 84 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
5340	2024-12-17T21:36:55.938Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5341	2024-12-17T21:36:55.938Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5342	2024-12-17T21:36:55.938Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5343	2024-12-17T21:36:55.939Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5344	2024-12-17T21:36:55.940Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5345	2024-12-17T21:36:55.941Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5346	2024-12-17T21:36:55.944Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
5347	2024-12-17T21:36:55.944Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5348	2024-12-17T21:36:55.944Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5349	2024-12-17T21:36:55.953Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
5350	2024-12-17T21:36:55.953Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.530009ms iteration = 84 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
5351	2024-12-17T21:37:00.405Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 432 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = e6744d4d-34f7-4467-9c10-8fdfe637e30b response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
5352	2024-12-17T21:37:05.932Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 15 reason = Timeout
5353	2024-12-17T21:37:05.932Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 16 reason = Timeout
5354	2024-12-17T21:37:05.932Z	WARN	nexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5
5355	2024-12-17T21:37:05.932Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 13.19\u{b5}s iteration = 16 status = {"enabled":false,"target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5"}
5356	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 29 reason = Timeout
5357	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 29 reason = Timeout
5358	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5359	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5360	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 17 reason = Timeout
5361	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 31 reason = Timeout
5362	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 15 reason = Timeout
5363	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
5364	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5365	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5366	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 29 reason = Timeout
5367	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 29 reason = Timeout
5368	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5369	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 29 reason = Timeout
5370	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 15 reason = Timeout
5371	2024-12-17T21:37:05.933Z	INFO	nexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
5372	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
5373	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 15 reason = Timeout
5374	2024-12-17T21:37:05.933Z	INFO	nexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-17 21:27:05.933050374 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
5375	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
5376	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 29 reason = Timeout
5377	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
5378	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
5379	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 85 reason = Timeout
5380	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5381	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 17 reason = Timeout
5382	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 15 reason = Timeout
5383	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = external_endpoints iteration = 15 reason = Timeout
5384	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5385	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 29 reason = Timeout
5386	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5387	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 15 reason = Timeout
5388	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
5389	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5390	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5391	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5392	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5393	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
5394	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5395	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
5396	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5397	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
5398	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5399	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
5400	2024-12-17T21:37:05.933Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0005c310-624c-4312-b0fd-35f571746851.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6ac4d5e8-fcd9-4aa9-aed5-fb3dc293aac2.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9b804364-993d-47ba-81a1-d0ef32f96529.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("9b804364-993d-47ba-81a1-d0ef32f96529.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 2084, tv_nsec: 67848172 } })
5401	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5402	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5403	2024-12-17T21:37:05.933Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("5dff2793-3ce8-4251-ba71-1a446966cbf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a761a47e-08b9-41c4-9bd8-8d77651dfa62.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a761a47e-08b9-41c4-9bd8-8d77651dfa62.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 2084, tv_nsec: 67890633 } })
5404	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5405	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 29 reason = Timeout
5406	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 29 reason = Timeout
5407	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 15 reason = Timeout
5408	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
5409	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 29 reason = Timeout
5410	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
5411	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
5412	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5413	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 29 reason = Timeout
5414	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
5415	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 29 reason = Timeout
5416	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
5417	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 15 reason = Timeout
5418	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5419	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
5420	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 15 reason = Timeout
5421	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
5422	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 29 reason = Timeout
5423	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 29 reason = Timeout
5424	2024-12-17T21:37:05.933Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
5425	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 85 reason = Timeout
5426	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5427	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5428	2024-12-17T21:37:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5429	2024-12-17T21:37:05.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
5430	2024-12-17T21:37:05.934Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
5431	2024-12-17T21:37:05.934Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.005187ms iteration = 15 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
5432	2024-12-17T21:37:05.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
5433	2024-12-17T21:37:05.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5434	2024-12-17T21:37:05.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
5435	2024-12-17T21:37:05.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
5436	2024-12-17T21:37:05.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
5437	2024-12-17T21:37:05.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
5438	2024-12-17T21:37:05.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
5439	2024-12-17T21:37:05.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5440	2024-12-17T21:37:05.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5441	2024-12-17T21:37:05.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
5442	2024-12-17T21:37:05.934Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
5443	2024-12-17T21:37:05.934Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.359467ms iteration = 15 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
5444	2024-12-17T21:37:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5445	2024-12-17T21:37:05.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5446	2024-12-17T21:37:05.936Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.138185ms iteration = 29 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
5447	2024-12-17T21:37:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5448	2024-12-17T21:37:05.937Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5449	2024-12-17T21:37:05.937Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5450	2024-12-17T21:37:05.939Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5451	2024-12-17T21:37:05.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
5452	2024-12-17T21:37:05.940Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
5453	2024-12-17T21:37:05.940Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
5454	2024-12-17T21:37:05.940Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
5455	2024-12-17T21:37:05.940Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5456	2024-12-17T21:37:05.940Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5457	2024-12-17T21:37:05.941Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
5458	2024-12-17T21:37:05.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5459	2024-12-17T21:37:05.941Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5460	2024-12-17T21:37:05.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
5461	2024-12-17T21:37:05.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
5462	2024-12-17T21:37:05.941Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5463	2024-12-17T21:37:05.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5464	2024-12-17T21:37:05.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5465	2024-12-17T21:37:05.942Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5466	2024-12-17T21:37:05.942Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5467	2024-12-17T21:37:05.942Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5468	2024-12-17T21:37:05.942Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2084, tv_nsec: 76792154 } })
5469	2024-12-17T21:37:05.942Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
5470	2024-12-17T21:37:05.943Z	INFO	nexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-17 21:27:05.933050374 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
5471	2024-12-17T21:37:05.943Z	DEBG	nexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 10.529285ms iteration = 15 status = {"expiration":"2024-12-17T21:27:05.933050374Z","pruned":[]}
5472	2024-12-17T21:37:05.944Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5473	2024-12-17T21:37:05.944Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2084, tv_nsec: 78638574 } })
5474	2024-12-17T21:37:05.944Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
5475	2024-12-17T21:37:05.944Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
5476	2024-12-17T21:37:05.944Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5477	2024-12-17T21:37:05.944Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5478	2024-12-17T21:37:05.945Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
5479	2024-12-17T21:37:05.945Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5480	2024-12-17T21:37:05.946Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5481	2024-12-17T21:37:05.947Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 13.944028ms iteration = 29 status = {"errors":[],"garbage_collect_requested":[]}
5482	2024-12-17T21:37:05.948Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
5483	2024-12-17T21:37:05.948Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
5484	2024-12-17T21:37:05.948Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5485	2024-12-17T21:37:05.948Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
5486	2024-12-17T21:37:05.948Z	DEBG	nexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
5487	2024-12-17T21:37:05.948Z	INFO	nexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
5488	2024-12-17T21:37:05.948Z	DEBG	nexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 15.036887ms iteration = 15 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
5489	2024-12-17T21:37:05.949Z	DEBG	nexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 15.987073ms iteration = 15 status = {"errors":[],"found_port_ok":[]}
5490	2024-12-17T21:37:05.949Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5491	2024-12-17T21:37:05.950Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5492	2024-12-17T21:37:05.950Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5493	2024-12-17T21:37:05.951Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
5494	2024-12-17T21:37:05.951Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5495	2024-12-17T21:37:05.951Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5496	2024-12-17T21:37:05.953Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5497	2024-12-17T21:37:05.954Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5498	2024-12-17T21:37:05.955Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5499	2024-12-17T21:37:05.955Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5500	2024-12-17T21:37:05.955Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5501	2024-12-17T21:37:05.957Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
5502	2024-12-17T21:37:05.958Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
5503	2024-12-17T21:37:05.958Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
5504	2024-12-17T21:37:05.958Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
5505	2024-12-17T21:37:05.958Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5506	2024-12-17T21:37:05.958Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5507	2024-12-17T21:37:05.959Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5508	2024-12-17T21:37:05.959Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2084, tv_nsec: 93522907 } })
5509	2024-12-17T21:37:05.959Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
5510	2024-12-17T21:37:05.960Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5511	2024-12-17T21:37:05.960Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5512	2024-12-17T21:37:05.961Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 27.211656ms iteration = 85 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
5513	2024-12-17T21:37:05.961Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
5514	2024-12-17T21:37:05.961Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
5515	2024-12-17T21:37:05.961Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5516	2024-12-17T21:37:05.961Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
5517	2024-12-17T21:37:05.962Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5518	2024-12-17T21:37:05.962Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5519	2024-12-17T21:37:05.962Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
5520	2024-12-17T21:37:05.962Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5521	2024-12-17T21:37:05.966Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
5522	2024-12-17T21:37:05.966Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5523	2024-12-17T21:37:05.966Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5524	2024-12-17T21:37:05.966Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
5525	2024-12-17T21:37:05.966Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5526	2024-12-17T21:37:05.966Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5527	2024-12-17T21:37:05.967Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
5528	2024-12-17T21:37:05.967Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5529	2024-12-17T21:37:05.967Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5530	2024-12-17T21:37:05.970Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5531	2024-12-17T21:37:05.970Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5532	2024-12-17T21:37:05.972Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5533	2024-12-17T21:37:05.977Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
5534	2024-12-17T21:37:05.977Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5535	2024-12-17T21:37:05.977Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5536	2024-12-17T21:37:05.983Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
5537	2024-12-17T21:37:05.983Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
5538	2024-12-17T21:37:05.983Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 50.234069ms iteration = 85 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
5539	2024-12-17T21:37:05.983Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
5540	2024-12-17T21:37:05.984Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5541	2024-12-17T21:37:05.985Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5542	2024-12-17T21:37:05.986Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5543	2024-12-17T21:37:05.988Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
5544	2024-12-17T21:37:05.989Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5545	2024-12-17T21:37:05.989Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
5546	2024-12-17T21:37:05.989Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 56.088698ms iteration = 29 status = {"errors":[],"records_set_to_done":[]}
5547	2024-12-17T21:37:05.991Z	DEBG	nexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
5548	2024-12-17T21:37:05.991Z	INFO	nexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
5549	2024-12-17T21:37:05.991Z	DEBG	nexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 58.091722ms iteration = 15 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
5550	2024-12-17T21:37:05.997Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 64.708422ms iteration = 29 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
5551	2024-12-17T21:37:05.998Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5552	2024-12-17T21:37:05.999Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5553	2024-12-17T21:37:05.999Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
5554	2024-12-17T21:37:05.999Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
5555	2024-12-17T21:37:05.999Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
5556	2024-12-17T21:37:05.999Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5557	2024-12-17T21:37:06.000Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5558	2024-12-17T21:37:06.001Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
5559	2024-12-17T21:37:06.001Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
5560	2024-12-17T21:37:06.001Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 67.293202ms iteration = 29 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
5561	2024-12-17T21:37:06.011Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5562	2024-12-17T21:37:06.012Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5563	2024-12-17T21:37:06.012Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
5564	2024-12-17T21:37:06.012Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
5565	2024-12-17T21:37:06.017Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5566	2024-12-17T21:37:06.018Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5567	2024-12-17T21:37:06.018Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5568	2024-12-17T21:37:06.020Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
5569	2024-12-17T21:37:06.020Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5570	2024-12-17T21:37:06.020Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5571	2024-12-17T21:37:06.074Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 141.446028ms iteration = 31 status = {"physical_disks_added":0}
5572	2024-12-17T21:37:06.076Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
5573	2024-12-17T21:37:06.102Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
5574	2024-12-17T21:37:06.117Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
5575	2024-12-17T21:37:06.117Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5576	2024-12-17T21:37:06.117Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "784c082b-bacd-4639-b4a2-ac0fae89f7c9", "content-length": "26", "date": "Tue, 17 Dec 2024 21:37:06 GMT"} })
5577	2024-12-17T21:37:06.117Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5578	2024-12-17T21:37:06.120Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
5579	2024-12-17T21:37:06.142Z	INFO	nexus (BackgroundTasks): VPC route manager sled d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
5580	2024-12-17T21:37:06.142Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
5581	2024-12-17T21:37:06.142Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "624f025e-cfc6-47a5-bbb2-2b85bffea398", "content-length": "372", "date": "Tue, 17 Dec 2024 21:37:06 GMT"} })
5582	2024-12-17T21:37:06.142Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5583	2024-12-17T21:37:06.143Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5584	2024-12-17T21:37:06.144Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 210.972089ms iteration = 29 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
5585	2024-12-17T21:37:06.145Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5586	2024-12-17T21:37:06.146Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5587	2024-12-17T21:37:06.146Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5588	2024-12-17T21:37:06.147Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5589	2024-12-17T21:37:06.148Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
5590	2024-12-17T21:37:06.148Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5591	2024-12-17T21:37:06.148Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5592	2024-12-17T21:37:06.149Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
5593	2024-12-17T21:37:06.149Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5594	2024-12-17T21:37:06.150Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5595	2024-12-17T21:37:06.179Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5596	2024-12-17T21:37:06.179Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5597	2024-12-17T21:37:06.185Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
5598	2024-12-17T21:37:06.185Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 251.566818ms iteration = 29 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
5599	2024-12-17T21:37:06.186Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5600	2024-12-17T21:37:06.188Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
5601	2024-12-17T21:37:06.188Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 574f0c35-b8f2-48fe-831c-78afc4a62aaa dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
5602	2024-12-17T21:37:06.188Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5603	2024-12-17T21:37:06.188Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5604	2024-12-17T21:37:06.189Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
5605	2024-12-17T21:37:06.189Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = ea6acdde-77d4-4bf6-b578-d9a0a29ac37b dns_zone_name = oxide.test found_so_far = 0 version = 2
5606	2024-12-17T21:37:06.189Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5607	2024-12-17T21:37:06.190Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5608	2024-12-17T21:37:06.191Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: d3278cd4-2587-4f0f-89dd-b4d4669a6ffe, lookup_type: ById(d3278cd4-2587-4f0f-89dd-b4d4669a6ffe) } result = Ok(())
5609	2024-12-17T21:37:06.191Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5610	2024-12-17T21:37:06.191Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5611	2024-12-17T21:37:06.193Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 260.787818ms iteration = 29 status = 0
5612	2024-12-17T21:37:06.202Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
5613	2024-12-17T21:37:06.202Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
5614	2024-12-17T21:37:06.202Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "be511cba-d454-4398-ab70-0e182fbb50b9", "date": "Tue, 17 Dec 2024 21:37:06 GMT"} })
5615	2024-12-17T21:37:06.202Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5616	2024-12-17T21:37:06.203Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5617	2024-12-17T21:37:06.205Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5618	2024-12-17T21:37:06.205Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5619	2024-12-17T21:37:06.205Z	DEBG	nexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-17T21:23:03.821125Z, time_modified: 2024-12-17T21:23:03.821125Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 81e78e1e-e55a-44f9-abdc-c9c916e1d9fd, db_silo: Silo { identity: SiloIdentity { id: 81e78e1e-e55a-44f9-abdc-c9c916e1d9fd, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-17T21:23:49.443930Z, time_modified: 2024-12-17T21:23:49.443930Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "e275643b21ab02733b9616c9cd3b0193e27c7f6294b7d310d6f1726c72768e50" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 81e78e1e-e55a-44f9-abdc-c9c916e1d9fd, db_silo: Silo { identity: SiloIdentity { id: 81e78e1e-e55a-44f9-abdc-c9c916e1d9fd, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-17T21:23:49.443930Z, time_modified: 2024-12-17T21:23:49.443930Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "e275643b21ab02733b9616c9cd3b0193e27c7f6294b7d310d6f1726c72768e50" }] }) }
5620	2024-12-17T21:37:06.205Z	DEBG	nexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 272.248518ms iteration = 15 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"81e78e1e-e55a-44f9-abdc-c9c916e1d9fd","tls_certs":["e275643b21ab02733b9616c9cd3b0193e27c7f6294b7d310d6f1726c72768e50"]}},"default_endpoint":{"silo_id":"81e78e1e-e55a-44f9-abdc-c9c916e1d9fd","tls_certs":["e275643b21ab02733b9616c9cd3b0193e27c7f6294b7d310d6f1726c72768e50"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
5621	2024-12-17T21:37:06.208Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5622	2024-12-17T21:37:06.208Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5623	2024-12-17T21:37:06.213Z	INFO	nexus (BackgroundTasks): VPC route manager sled d3278cd4-2587-4f0f-89dd-b4d4669a6ffe push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
5624	2024-12-17T21:37:06.213Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
5625	2024-12-17T21:37:06.214Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "17a4ff93-7c8f-4648-abb4-d42eafb7f52d", "date": "Tue, 17 Dec 2024 21:37:06 GMT"} })
5626	2024-12-17T21:37:06.216Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 282.485926ms iteration = 29 status = {}
5627	2024-12-17T21:37:06.218Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 284.419767ms iteration = 29 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
5628	2024-12-17T21:37:06.227Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5629	2024-12-17T21:37:06.230Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
5630	2024-12-17T21:37:06.230Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5631	2024-12-17T21:37:06.230Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5632	2024-12-17T21:37:06.233Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 574f0c35-b8f2-48fe-831c-78afc4a62aaa dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
5633	2024-12-17T21:37:06.233Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
5634	2024-12-17T21:37:06.233Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-17 21:23:48.446255 UTC dns_group = internal generation = 1
5635	2024-12-17T21:37:06.233Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 299.886665ms iteration = 15 status = {"generation":1}
5636	2024-12-17T21:37:06.235Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
5637	2024-12-17T21:37:06.236Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5638	2024-12-17T21:37:06.237Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
5639	2024-12-17T21:37:06.237Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5640	2024-12-17T21:37:06.237Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e28435e-1e9c-43a1-811f-7815331de5b4", "content-length": "26", "date": "Tue, 17 Dec 2024 21:37:06 GMT"} })
5641	2024-12-17T21:37:06.237Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5642	2024-12-17T21:37:06.239Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
5643	2024-12-17T21:37:06.239Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5644	2024-12-17T21:37:06.239Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5645	2024-12-17T21:37:06.239Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
5646	2024-12-17T21:37:06.241Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = ea6acdde-77d4-4bf6-b578-d9a0a29ac37b dns_zone_name = oxide.test found_so_far = 1 version = 2
5647	2024-12-17T21:37:06.241Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
5648	2024-12-17T21:37:06.241Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-17 21:23:49.491999 UTC dns_group = external generation = 2
5649	2024-12-17T21:37:06.241Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 308.360755ms iteration = 15 status = {"generation":2}
5650	2024-12-17T21:37:06.253Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
5651	2024-12-17T21:37:06.253Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "eb1db116-be16-4f00-91d1-bba9bd78acca", "date": "Tue, 17 Dec 2024 21:37:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
5652	2024-12-17T21:37:06.253Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
5653	2024-12-17T21:37:06.254Z	WARN	nexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
5654	2024-12-17T21:37:06.254Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 320.636407ms iteration = 15 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
5655	2024-12-17T21:37:06.255Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5656	2024-12-17T21:37:06.255Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "bbc661e9-e447-461c-98b9-11e47e29b83b", "date": "Tue, 17 Dec 2024 21:37:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
5657	2024-12-17T21:37:06.255Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
5658	2024-12-17T21:37:06.255Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5659	2024-12-17T21:37:06.255Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 zone_address = fd00:1122:3344:101::2
5660	2024-12-17T21:37:06.255Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5661	2024-12-17T21:37:06.256Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1a81c12-a029-4e40-a954-05e2ef7dc867", "content-length": "26", "date": "Tue, 17 Dec 2024 21:37:06 GMT"} })
5662	2024-12-17T21:37:06.256Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5663	2024-12-17T21:37:06.257Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5664	2024-12-17T21:37:06.258Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
5665	2024-12-17T21:37:06.258Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
5666	2024-12-17T21:37:06.258Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 325.903391ms iteration = 17 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
5667	2024-12-17T21:37:06.259Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
5668	2024-12-17T21:37:06.259Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5669	2024-12-17T21:37:06.259Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5670	2024-12-17T21:37:06.267Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 334.257356ms iteration = 29 status = {}
5671	2024-12-17T21:37:06.287Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5672	2024-12-17T21:37:06.289Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
5673	2024-12-17T21:37:06.289Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5674	2024-12-17T21:37:06.289Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5675	2024-12-17T21:37:06.339Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
5676	2024-12-17T21:37:06.340Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "75d7fa2b-f9af-4e76-a8ee-f96ad1eae766", "date": "Tue, 17 Dec 2024 21:37:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
5677	2024-12-17T21:37:06.340Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
5678	2024-12-17T21:37:06.341Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
5679	2024-12-17T21:37:06.342Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e5099b4-0dd7-4f1d-a41b-40f0fcad160c", "content-length": "2", "date": "Tue, 17 Dec 2024 21:37:05 GMT"} })
5680	2024-12-17T21:37:06.342Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "299e0e5a-d377-4f48-a729-c84d34e4463a", "date": "Tue, 17 Dec 2024 21:37:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
5681	2024-12-17T21:37:06.342Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
5682	2024-12-17T21:37:06.344Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5683	2024-12-17T21:37:06.344Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5684	2024-12-17T21:37:06.345Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "4122525e-3dc0-4aa3-9aaf-d24e380ebcd9", "date": "Tue, 17 Dec 2024 21:37:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
5685	2024-12-17T21:37:06.345Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
5686	2024-12-17T21:37:06.346Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 413.46021ms iteration = 29 status = null
5687	2024-12-17T21:37:06.347Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
5688	2024-12-17T21:37:06.347Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 414.439057ms iteration = 17 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
5689	2024-12-17T21:37:06.414Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
5690	2024-12-17T21:37:06.440Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5691	2024-12-17T21:37:06.440Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5692	2024-12-17T21:37:06.441Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "79aaa42a-7ccf-4032-8e46-8f49d7825a66", "content-length": "691", "date": "Tue, 17 Dec 2024 21:37:06 GMT"} })
5693	2024-12-17T21:37:06.441Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
5694	2024-12-17T21:37:06.441Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
5695	2024-12-17T21:37:06.441Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
5696	2024-12-17T21:37:06.442Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 509.898129ms iteration = 29 status = {}
5697	2024-12-17T21:37:06.442Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 1a3e1a79-c78b-4fb4-921b-c5e1e32e8b40, rack_id: 8216c8d1-8011-434d-a57a-2b1ce2e89ad8, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(d6c2617a-6b89-4779-bc70-3002fb0a7e5e) } rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0
5698	2024-12-17T21:37:06.442Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5699	2024-12-17T21:37:06.443Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5700	2024-12-17T21:37:06.452Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
5701	2024-12-17T21:37:06.453Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "62605d63-c323-470e-a2cd-fc81969fbbc8", "content-length": "134", "date": "Tue, 17 Dec 2024 21:37:06 GMT"} })
5702	2024-12-17T21:37:06.453Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
5703	2024-12-17T21:37:06.453Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = d6c2617a-6b89-4779-bc70-3002fb0a7e5e
5704	2024-12-17T21:37:06.453Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
5705	2024-12-17T21:37:06.453Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
5706	2024-12-17T21:37:06.453Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5707	2024-12-17T21:37:06.453Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5708	2024-12-17T21:37:06.453Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5709	2024-12-17T21:37:06.453Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
5710	2024-12-17T21:37:06.453Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5711	2024-12-17T21:37:06.453Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
5712	2024-12-17T21:37:06.454Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
5713	2024-12-17T21:37:06.454Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5714	2024-12-17T21:37:06.454Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5715	2024-12-17T21:37:06.455Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
5716	2024-12-17T21:37:06.455Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "88ef2d02-3b3d-44f1-a45e-f43b59e318b4", "content-length": "2", "date": "Tue, 17 Dec 2024 21:37:06 GMT"} })
5717	2024-12-17T21:37:06.455Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
5718	2024-12-17T21:37:06.456Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "61e652b5-b1d8-47f6-a053-e1afb2c22bb5", "content-length": "51", "date": "Tue, 17 Dec 2024 21:37:06 GMT"} })
5719	2024-12-17T21:37:06.456Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0
5720	2024-12-17T21:37:06.456Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
5721	2024-12-17T21:37:06.548Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "1823176c-e69e-4b46-bb55-5ec5f8e6d2c8", "date": "Tue, 17 Dec 2024 21:37:06 GMT"} })
5722	2024-12-17T21:37:06.548Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch = Switch0
5723	2024-12-17T21:37:06.548Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
5724	2024-12-17T21:37:06.549Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "789701f3-ae35-415f-8f63-14826eb1a3ce", "content-length": "682", "date": "Tue, 17 Dec 2024 21:37:06 GMT"} })
5725	2024-12-17T21:37:06.549Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5726	2024-12-17T21:37:06.549Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5727	2024-12-17T21:37:06.553Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5728	2024-12-17T21:37:06.553Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5729	2024-12-17T21:37:06.557Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5730	2024-12-17T21:37:06.557Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5731	2024-12-17T21:37:06.559Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5732	2024-12-17T21:37:06.559Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5733	2024-12-17T21:37:06.566Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 633.917216ms iteration = 29 status = {}
5734	2024-12-17T21:37:10.404Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 293 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = 12ab9417-7b27-488d-9cf2-8011df365a9b response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
5735	2024-12-17T21:37:15.933Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 86 reason = Timeout
5736	2024-12-17T21:37:15.933Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5737	2024-12-17T21:37:15.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5738	2024-12-17T21:37:15.933Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 86 reason = Timeout
5739	2024-12-17T21:37:15.933Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5740	2024-12-17T21:37:15.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5741	2024-12-17T21:37:15.935Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5742	2024-12-17T21:37:15.938Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5743	2024-12-17T21:37:15.938Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5744	2024-12-17T21:37:15.938Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5745	2024-12-17T21:37:15.940Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5746	2024-12-17T21:37:15.940Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5747	2024-12-17T21:37:15.942Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5748	2024-12-17T21:37:15.945Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
5749	2024-12-17T21:37:15.945Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5750	2024-12-17T21:37:15.945Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5751	2024-12-17T21:37:15.956Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
5752	2024-12-17T21:37:15.956Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.712582ms iteration = 86 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
5753	2024-12-17T21:37:15.962Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5754	2024-12-17T21:37:15.962Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5755	2024-12-17T21:37:15.991Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 57.63953ms iteration = 86 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
5756	2024-12-17T21:37:20.404Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 459 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = d06c0de8-3751-4e17-821c-a1b4334262b8 response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
5757	2024-12-17T21:37:25.933Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 87 reason = Timeout
5758	2024-12-17T21:37:25.933Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5759	2024-12-17T21:37:25.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5760	2024-12-17T21:37:25.933Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 87 reason = Timeout
5761	2024-12-17T21:37:25.933Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5762	2024-12-17T21:37:25.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5763	2024-12-17T21:37:25.970Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5764	2024-12-17T21:37:25.970Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5765	2024-12-17T21:37:25.971Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5766	2024-12-17T21:37:25.972Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 38.537823ms iteration = 87 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
5767	2024-12-17T21:37:25.974Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5768	2024-12-17T21:37:25.974Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5769	2024-12-17T21:37:25.974Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5770	2024-12-17T21:37:25.976Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5771	2024-12-17T21:37:25.976Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5772	2024-12-17T21:37:25.978Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5773	2024-12-17T21:37:25.980Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
5774	2024-12-17T21:37:25.980Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5775	2024-12-17T21:37:25.980Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5776	2024-12-17T21:37:25.991Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
5777	2024-12-17T21:37:25.991Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 58.061972ms iteration = 87 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
5778	2024-12-17T21:37:30.404Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 437 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = 1b9dce9d-871d-4975-a886-bca56e142e85 response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
5779	2024-12-17T21:37:35.933Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 88 reason = Timeout
5780	2024-12-17T21:37:35.933Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 30 reason = Timeout
5781	2024-12-17T21:37:35.933Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5782	2024-12-17T21:37:35.933Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5783	2024-12-17T21:37:35.933Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 30 reason = Timeout
5784	2024-12-17T21:37:35.933Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 30 reason = Timeout
5785	2024-12-17T21:37:35.933Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
5786	2024-12-17T21:37:35.933Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5787	2024-12-17T21:37:35.933Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 30 reason = Timeout
5788	2024-12-17T21:37:35.933Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5789	2024-12-17T21:37:35.933Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 30 reason = Timeout
5790	2024-12-17T21:37:35.933Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 30 reason = Timeout
5791	2024-12-17T21:37:35.933Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
5792	2024-12-17T21:37:35.933Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 30 reason = Timeout
5793	2024-12-17T21:37:35.933Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5794	2024-12-17T21:37:35.933Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 32 reason = Timeout
5795	2024-12-17T21:37:35.933Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5796	2024-12-17T21:37:35.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5797	2024-12-17T21:37:35.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5798	2024-12-17T21:37:35.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
5799	2024-12-17T21:37:35.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
5800	2024-12-17T21:37:35.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5801	2024-12-17T21:37:35.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5802	2024-12-17T21:37:35.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5803	2024-12-17T21:37:35.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5804	2024-12-17T21:37:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 30 reason = Timeout
5805	2024-12-17T21:37:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 30 reason = Timeout
5806	2024-12-17T21:37:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
5807	2024-12-17T21:37:35.934Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
5808	2024-12-17T21:37:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5809	2024-12-17T21:37:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 30 reason = Timeout
5810	2024-12-17T21:37:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5811	2024-12-17T21:37:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 30 reason = Timeout
5812	2024-12-17T21:37:35.934Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
5813	2024-12-17T21:37:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 30 reason = Timeout
5814	2024-12-17T21:37:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
5815	2024-12-17T21:37:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5816	2024-12-17T21:37:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 30 reason = Timeout
5817	2024-12-17T21:37:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
5818	2024-12-17T21:37:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 30 reason = Timeout
5819	2024-12-17T21:37:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5820	2024-12-17T21:37:35.934Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 88 reason = Timeout
5821	2024-12-17T21:37:35.934Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5822	2024-12-17T21:37:35.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
5823	2024-12-17T21:37:35.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5824	2024-12-17T21:37:35.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
5825	2024-12-17T21:37:35.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5826	2024-12-17T21:37:35.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
5827	2024-12-17T21:37:35.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
5828	2024-12-17T21:37:35.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
5829	2024-12-17T21:37:35.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5830	2024-12-17T21:37:35.935Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5831	2024-12-17T21:37:35.936Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2114, tv_nsec: 70893024 } })
5832	2024-12-17T21:37:35.936Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
5833	2024-12-17T21:37:35.936Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.776923ms iteration = 30 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
5834	2024-12-17T21:37:35.937Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5835	2024-12-17T21:37:35.938Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2114, tv_nsec: 72794936 } })
5836	2024-12-17T21:37:35.938Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
5837	2024-12-17T21:37:35.938Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5838	2024-12-17T21:37:35.938Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5839	2024-12-17T21:37:35.938Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5840	2024-12-17T21:37:35.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
5841	2024-12-17T21:37:35.939Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5842	2024-12-17T21:37:35.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5843	2024-12-17T21:37:35.941Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5844	2024-12-17T21:37:35.942Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 8.14042ms iteration = 30 status = {"errors":[],"garbage_collect_requested":[]}
5845	2024-12-17T21:37:35.943Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
5846	2024-12-17T21:37:35.943Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5847	2024-12-17T21:37:35.943Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5848	2024-12-17T21:37:35.947Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5849	2024-12-17T21:37:35.947Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5850	2024-12-17T21:37:35.948Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5851	2024-12-17T21:37:35.948Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5852	2024-12-17T21:37:35.950Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2114, tv_nsec: 85026827 } })
5853	2024-12-17T21:37:35.950Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
5854	2024-12-17T21:37:35.951Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5855	2024-12-17T21:37:35.951Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5856	2024-12-17T21:37:35.971Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
5857	2024-12-17T21:37:35.971Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
5858	2024-12-17T21:37:35.971Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5859	2024-12-17T21:37:35.971Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5860	2024-12-17T21:37:35.971Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
5861	2024-12-17T21:37:35.971Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5862	2024-12-17T21:37:35.972Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
5863	2024-12-17T21:37:35.972Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
5864	2024-12-17T21:37:35.973Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5865	2024-12-17T21:37:35.974Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5866	2024-12-17T21:37:35.974Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 41.849382ms iteration = 30 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
5867	2024-12-17T21:37:35.975Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
5868	2024-12-17T21:37:35.975Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
5869	2024-12-17T21:37:35.975Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5870	2024-12-17T21:37:35.975Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
5871	2024-12-17T21:37:35.976Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
5872	2024-12-17T21:37:35.976Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5873	2024-12-17T21:37:35.976Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 42.648245ms iteration = 30 status = {"errors":[],"records_set_to_done":[]}
5874	2024-12-17T21:37:35.976Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5875	2024-12-17T21:37:35.977Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
5876	2024-12-17T21:37:35.977Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5877	2024-12-17T21:37:35.978Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5878	2024-12-17T21:37:35.979Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5879	2024-12-17T21:37:35.980Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5880	2024-12-17T21:37:35.980Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5881	2024-12-17T21:37:35.980Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5882	2024-12-17T21:37:35.981Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5883	2024-12-17T21:37:35.981Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5884	2024-12-17T21:37:35.982Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
5885	2024-12-17T21:37:35.982Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5886	2024-12-17T21:37:35.982Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5887	2024-12-17T21:37:35.983Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
5888	2024-12-17T21:37:35.983Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5889	2024-12-17T21:37:35.983Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
5890	2024-12-17T21:37:35.984Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
5891	2024-12-17T21:37:35.984Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5892	2024-12-17T21:37:35.984Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
5893	2024-12-17T21:37:35.984Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5894	2024-12-17T21:37:35.984Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5895	2024-12-17T21:37:35.984Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5896	2024-12-17T21:37:35.984Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5897	2024-12-17T21:37:35.984Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5898	2024-12-17T21:37:35.987Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
5899	2024-12-17T21:37:35.988Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
5900	2024-12-17T21:37:35.999Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
5901	2024-12-17T21:37:35.999Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 66.4794ms iteration = 88 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
5902	2024-12-17T21:37:36.005Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5903	2024-12-17T21:37:36.005Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5904	2024-12-17T21:37:36.006Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5905	2024-12-17T21:37:36.006Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5906	2024-12-17T21:37:36.008Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5907	2024-12-17T21:37:36.009Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5908	2024-12-17T21:37:36.010Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
5909	2024-12-17T21:37:36.010Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5910	2024-12-17T21:37:36.011Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9359dc1e-bf04-47cc-ac13-692740910b0d", "content-length": "26", "date": "Tue, 17 Dec 2024 21:37:35 GMT"} })
5911	2024-12-17T21:37:36.011Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5912	2024-12-17T21:37:36.013Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
5913	2024-12-17T21:37:36.023Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 89.297496ms iteration = 88 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
5914	2024-12-17T21:37:36.024Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
5915	2024-12-17T21:37:36.024Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 89.766119ms iteration = 30 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
5916	2024-12-17T21:37:36.024Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
5917	2024-12-17T21:37:36.024Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5918	2024-12-17T21:37:36.024Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9cd8478-4bbb-4b12-8748-77f9da722820", "content-length": "26", "date": "Tue, 17 Dec 2024 21:37:35 GMT"} })
5919	2024-12-17T21:37:36.024Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5920	2024-12-17T21:37:36.026Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
5921	2024-12-17T21:37:36.027Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 zone_address = fd00:1122:3344:101::2
5922	2024-12-17T21:37:36.027Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5923	2024-12-17T21:37:36.027Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7182a76f-e75e-4460-a2f4-6c5ece11491b", "content-length": "26", "date": "Tue, 17 Dec 2024 21:37:35 GMT"} })
5924	2024-12-17T21:37:36.027Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5925	2024-12-17T21:37:36.029Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
5926	2024-12-17T21:37:36.029Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 95.468234ms iteration = 30 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
5927	2024-12-17T21:37:36.029Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
5928	2024-12-17T21:37:36.029Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
5929	2024-12-17T21:37:36.029Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 95.679079ms iteration = 30 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
5930	2024-12-17T21:37:36.039Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 104.93188ms iteration = 30 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
5931	2024-12-17T21:37:36.040Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5932	2024-12-17T21:37:36.042Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: d3278cd4-2587-4f0f-89dd-b4d4669a6ffe, lookup_type: ById(d3278cd4-2587-4f0f-89dd-b4d4669a6ffe) } result = Ok(())
5933	2024-12-17T21:37:36.042Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5934	2024-12-17T21:37:36.042Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5935	2024-12-17T21:37:36.048Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 115.017073ms iteration = 30 status = 0
5936	2024-12-17T21:37:36.062Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5937	2024-12-17T21:37:36.062Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5938	2024-12-17T21:37:36.092Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 159.65531ms iteration = 32 status = {"physical_disks_added":0}
5939	2024-12-17T21:37:36.093Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
5940	2024-12-17T21:37:36.093Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e24d91e-663b-4eb6-a8a6-e358c6bea989", "content-length": "2", "date": "Tue, 17 Dec 2024 21:37:35 GMT"} })
5941	2024-12-17T21:37:36.095Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5942	2024-12-17T21:37:36.096Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5943	2024-12-17T21:37:36.121Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 188.059859ms iteration = 30 status = {}
5944	2024-12-17T21:37:36.129Z	INFO	nexus (BackgroundTasks): VPC route manager sled d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
5945	2024-12-17T21:37:36.129Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
5946	2024-12-17T21:37:36.130Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5947	2024-12-17T21:37:36.131Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 198.448011ms iteration = 30 status = null
5948	2024-12-17T21:37:36.132Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
5949	2024-12-17T21:37:36.132Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5950	2024-12-17T21:37:36.132Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5951	2024-12-17T21:37:36.134Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5952	2024-12-17T21:37:36.135Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
5953	2024-12-17T21:37:36.135Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5954	2024-12-17T21:37:36.135Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5955	2024-12-17T21:37:36.179Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
5956	2024-12-17T21:37:36.187Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5957	2024-12-17T21:37:36.187Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5958	2024-12-17T21:37:36.188Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 1a3e1a79-c78b-4fb4-921b-c5e1e32e8b40, rack_id: 8216c8d1-8011-434d-a57a-2b1ce2e89ad8, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(d6c2617a-6b89-4779-bc70-3002fb0a7e5e) } rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0
5959	2024-12-17T21:37:36.188Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5960	2024-12-17T21:37:36.189Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5961	2024-12-17T21:37:36.199Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
5962	2024-12-17T21:37:36.199Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc4d3679-f34a-4c73-bd51-549b88c6960e", "content-length": "134", "date": "Tue, 17 Dec 2024 21:37:35 GMT"} })
5963	2024-12-17T21:37:36.199Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
5964	2024-12-17T21:37:36.199Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = d6c2617a-6b89-4779-bc70-3002fb0a7e5e
5965	2024-12-17T21:37:36.200Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
5966	2024-12-17T21:37:36.200Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
5967	2024-12-17T21:37:36.200Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5968	2024-12-17T21:37:36.200Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5969	2024-12-17T21:37:36.200Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5970	2024-12-17T21:37:36.200Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
5971	2024-12-17T21:37:36.200Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5972	2024-12-17T21:37:36.200Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
5973	2024-12-17T21:37:36.200Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
5974	2024-12-17T21:37:36.200Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5975	2024-12-17T21:37:36.201Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5976	2024-12-17T21:37:36.202Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
5977	2024-12-17T21:37:36.202Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f682672-51f8-425f-a1d9-76f837fc3d52", "content-length": "2", "date": "Tue, 17 Dec 2024 21:37:35 GMT"} })
5978	2024-12-17T21:37:36.202Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
5979	2024-12-17T21:37:36.203Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "1da79b04-93f3-42c4-9c30-c392c13f039b", "content-length": "51", "date": "Tue, 17 Dec 2024 21:37:35 GMT"} })
5980	2024-12-17T21:37:36.203Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0
5981	2024-12-17T21:37:36.203Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
5982	2024-12-17T21:37:36.209Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "be6d146d-7749-4ff9-af62-8d9b2a9a669e", "content-length": "372", "date": "Tue, 17 Dec 2024 21:37:35 GMT"} })
5983	2024-12-17T21:37:36.209Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5984	2024-12-17T21:37:36.209Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5985	2024-12-17T21:37:36.218Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "a01fda4b-ed97-40d8-9c59-90856555842b", "content-length": "691", "date": "Tue, 17 Dec 2024 21:37:36 GMT"} })
5986	2024-12-17T21:37:36.218Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
5987	2024-12-17T21:37:36.218Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
5988	2024-12-17T21:37:36.218Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
5989	2024-12-17T21:37:36.219Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 286.668329ms iteration = 30 status = {}
5990	2024-12-17T21:37:36.225Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
5991	2024-12-17T21:37:36.225Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
5992	2024-12-17T21:37:36.227Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "fb5eea55-f952-4e7d-ac19-7d2fe2018bc0", "date": "Tue, 17 Dec 2024 21:37:36 GMT"} })
5993	2024-12-17T21:37:36.227Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5994	2024-12-17T21:37:36.227Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5995	2024-12-17T21:37:36.229Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5996	2024-12-17T21:37:36.229Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5997	2024-12-17T21:37:36.232Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5998	2024-12-17T21:37:36.232Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5999	2024-12-17T21:37:36.237Z	INFO	nexus (BackgroundTasks): VPC route manager sled d3278cd4-2587-4f0f-89dd-b4d4669a6ffe push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
6000	2024-12-17T21:37:36.237Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
6001	2024-12-17T21:37:36.239Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "02bc9457-5c49-414d-8951-1819525f7368", "date": "Tue, 17 Dec 2024 21:37:36 GMT"} })
6002	2024-12-17T21:37:36.241Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 306.809664ms iteration = 30 status = {}
6003	2024-12-17T21:37:36.302Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "ed423654-cc8f-40ef-96e4-70612f807fef", "date": "Tue, 17 Dec 2024 21:37:36 GMT"} })
6004	2024-12-17T21:37:36.302Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch = Switch0
6005	2024-12-17T21:37:36.302Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
6006	2024-12-17T21:37:36.302Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac13c68f-86e2-401d-b266-399344e2aff9", "content-length": "682", "date": "Tue, 17 Dec 2024 21:37:36 GMT"} })
6007	2024-12-17T21:37:36.302Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6008	2024-12-17T21:37:36.302Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6009	2024-12-17T21:37:36.305Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6010	2024-12-17T21:37:36.306Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6011	2024-12-17T21:37:36.309Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6012	2024-12-17T21:37:36.309Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6013	2024-12-17T21:37:36.311Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6014	2024-12-17T21:37:36.311Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6015	2024-12-17T21:37:36.317Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 384.510876ms iteration = 30 status = {}
6016	2024-12-17T21:37:40.404Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 512 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = 0d06f1fd-5513-4dfd-bf15-e5b52d44dfc6 response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
6017	2024-12-17T21:37:45.933Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 89 reason = Timeout
6018	2024-12-17T21:37:45.933Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6019	2024-12-17T21:37:45.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6020	2024-12-17T21:37:45.933Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 89 reason = Timeout
6021	2024-12-17T21:37:45.933Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6022	2024-12-17T21:37:45.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6023	2024-12-17T21:37:45.935Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6024	2024-12-17T21:37:45.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6025	2024-12-17T21:37:45.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6026	2024-12-17T21:37:45.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6027	2024-12-17T21:37:45.937Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6028	2024-12-17T21:37:45.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6029	2024-12-17T21:37:45.939Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6030	2024-12-17T21:37:45.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6031	2024-12-17T21:37:45.941Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6032	2024-12-17T21:37:45.943Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
6033	2024-12-17T21:37:45.943Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6034	2024-12-17T21:37:45.943Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6035	2024-12-17T21:37:45.955Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
6036	2024-12-17T21:37:45.955Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.17531ms iteration = 89 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
6037	2024-12-17T21:37:45.964Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.006269ms iteration = 89 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6038	2024-12-17T21:37:50.404Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 447 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = 6441c890-0f4b-47ed-b32b-76b552bac3a3 response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
6039	2024-12-17T21:37:55.933Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 90 reason = Timeout
6040	2024-12-17T21:37:55.933Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6041	2024-12-17T21:37:55.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6042	2024-12-17T21:37:55.933Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 90 reason = Timeout
6043	2024-12-17T21:37:55.933Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6044	2024-12-17T21:37:55.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6045	2024-12-17T21:37:55.935Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6046	2024-12-17T21:37:55.936Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6047	2024-12-17T21:37:55.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6048	2024-12-17T21:37:55.937Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.227065ms iteration = 90 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6049	2024-12-17T21:37:55.938Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6050	2024-12-17T21:37:55.938Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6051	2024-12-17T21:37:55.938Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6052	2024-12-17T21:37:55.940Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6053	2024-12-17T21:37:55.940Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6054	2024-12-17T21:37:55.941Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6055	2024-12-17T21:37:55.944Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
6056	2024-12-17T21:37:55.944Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6057	2024-12-17T21:37:55.944Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6058	2024-12-17T21:37:55.955Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
6059	2024-12-17T21:37:55.955Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.768516ms iteration = 90 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
6060	2024-12-17T21:38:00.404Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 449 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = 8015c1ce-d22f-4c43-96c5-980abe927774 response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
6061	2024-12-17T21:38:05.932Z	DEBG	nexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 7 reason = Timeout
6062	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 16 reason = Timeout
6063	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
6064	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6065	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
6066	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 31 reason = Timeout
6067	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6068	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 18 reason = Timeout
6069	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 16 reason = Timeout
6070	2024-12-17T21:38:05.933Z	INFO	nexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
6071	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 31 reason = Timeout
6072	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
6073	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
6074	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 31 reason = Timeout
6075	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6076	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 16 reason = Timeout
6077	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
6078	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6079	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 31 reason = Timeout
6080	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
6081	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 16 reason = Timeout
6082	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 18 reason = Timeout
6083	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 33 reason = Timeout
6084	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = external_endpoints iteration = 16 reason = Timeout
6085	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6086	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6087	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 16 reason = Timeout
6088	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 31 reason = Timeout
6089	2024-12-17T21:38:05.933Z	INFO	nexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-17 21:28:05.933191171 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
6090	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
6091	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
6092	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 31 reason = Timeout
6093	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 31 reason = Timeout
6094	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6095	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 91 reason = Timeout
6096	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 16 reason = Timeout
6097	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6098	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 17 reason = Timeout
6099	2024-12-17T21:38:05.933Z	WARN	nexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5
6100	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 11.94\u{b5}s iteration = 17 status = {"enabled":false,"target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5"}
6101	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 31 reason = Timeout
6102	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 31 reason = Timeout
6103	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
6104	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
6105	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 31 reason = Timeout
6106	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 31 reason = Timeout
6107	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
6108	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
6109	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6110	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 4 reason = Timeout
6111	2024-12-17T21:38:05.933Z	DEBG	nexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
6112	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6113	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 31 reason = Timeout
6114	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
6115	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 31 reason = Timeout
6116	2024-12-17T21:38:05.933Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
6117	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6118	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 91 reason = Timeout
6119	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6120	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 31 reason = Timeout
6121	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6122	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 16 reason = Timeout
6123	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
6124	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 16 reason = Timeout
6125	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
6126	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 16 reason = Timeout
6127	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
6128	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6129	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
6130	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
6131	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6132	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6133	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6134	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
6135	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
6136	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6137	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6138	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
6139	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6140	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
6141	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
6142	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6143	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6144	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
6145	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6146	2024-12-17T21:38:05.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6147	2024-12-17T21:38:05.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6148	2024-12-17T21:38:05.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
6149	2024-12-17T21:38:05.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
6150	2024-12-17T21:38:05.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
6151	2024-12-17T21:38:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
6152	2024-12-17T21:38:05.934Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("5dff2793-3ce8-4251-ba71-1a446966cbf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a761a47e-08b9-41c4-9bd8-8d77651dfa62.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a761a47e-08b9-41c4-9bd8-8d77651dfa62.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 2144, tv_nsec: 68734356 } })
6153	2024-12-17T21:38:05.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
6154	2024-12-17T21:38:05.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6155	2024-12-17T21:38:05.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
6156	2024-12-17T21:38:05.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6157	2024-12-17T21:38:05.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
6158	2024-12-17T21:38:05.934Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0005c310-624c-4312-b0fd-35f571746851.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6ac4d5e8-fcd9-4aa9-aed5-fb3dc293aac2.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9b804364-993d-47ba-81a1-d0ef32f96529.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("9b804364-993d-47ba-81a1-d0ef32f96529.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 2144, tv_nsec: 68761147 } })
6159	2024-12-17T21:38:05.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
6160	2024-12-17T21:38:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
6161	2024-12-17T21:38:05.934Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
6162	2024-12-17T21:38:05.934Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.281524ms iteration = 16 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
6163	2024-12-17T21:38:05.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
6164	2024-12-17T21:38:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
6165	2024-12-17T21:38:05.934Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
6166	2024-12-17T21:38:05.934Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.317456ms iteration = 16 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
6167	2024-12-17T21:38:05.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
6168	2024-12-17T21:38:05.934Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6169	2024-12-17T21:38:05.934Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6170	2024-12-17T21:38:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6171	2024-12-17T21:38:05.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6172	2024-12-17T21:38:05.936Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6173	2024-12-17T21:38:05.937Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.09047ms iteration = 31 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
6174	2024-12-17T21:38:05.938Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2144, tv_nsec: 73057383 } })
6175	2024-12-17T21:38:05.938Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
6176	2024-12-17T21:38:05.938Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
6177	2024-12-17T21:38:05.938Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
6178	2024-12-17T21:38:05.938Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
6179	2024-12-17T21:38:05.938Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6180	2024-12-17T21:38:05.940Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
6181	2024-12-17T21:38:05.940Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
6182	2024-12-17T21:38:05.941Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2144, tv_nsec: 75750336 } })
6183	2024-12-17T21:38:05.941Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
6184	2024-12-17T21:38:05.941Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
6185	2024-12-17T21:38:05.942Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 8.745546ms iteration = 31 status = {"errors":[],"records_set_to_done":[]}
6186	2024-12-17T21:38:05.942Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6187	2024-12-17T21:38:05.943Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6188	2024-12-17T21:38:05.944Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
6189	2024-12-17T21:38:05.944Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6190	2024-12-17T21:38:05.944Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6191	2024-12-17T21:38:05.945Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
6192	2024-12-17T21:38:05.945Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6193	2024-12-17T21:38:05.945Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6194	2024-12-17T21:38:05.946Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6195	2024-12-17T21:38:05.946Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6196	2024-12-17T21:38:05.946Z	INFO	nexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-17 21:28:05.933191171 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
6197	2024-12-17T21:38:05.946Z	DEBG	nexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 13.754003ms iteration = 16 status = {"expiration":"2024-12-17T21:28:05.933191171Z","pruned":[]}
6198	2024-12-17T21:38:05.947Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6199	2024-12-17T21:38:05.947Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6200	2024-12-17T21:38:05.948Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6201	2024-12-17T21:38:05.948Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6202	2024-12-17T21:38:05.948Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6203	2024-12-17T21:38:05.949Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
6204	2024-12-17T21:38:05.949Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6205	2024-12-17T21:38:05.949Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6206	2024-12-17T21:38:05.950Z	DEBG	nexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 17.388481ms iteration = 16 status = {"errors":[],"found_port_ok":[]}
6207	2024-12-17T21:38:05.950Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6208	2024-12-17T21:38:05.952Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6209	2024-12-17T21:38:05.952Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6210	2024-12-17T21:38:05.952Z	DEBG	nexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
6211	2024-12-17T21:38:05.952Z	INFO	nexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
6212	2024-12-17T21:38:05.952Z	DEBG	nexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 18.975554ms iteration = 16 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
6213	2024-12-17T21:38:05.953Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
6214	2024-12-17T21:38:05.953Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
6215	2024-12-17T21:38:05.954Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
6216	2024-12-17T21:38:05.954Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
6217	2024-12-17T21:38:05.954Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6218	2024-12-17T21:38:05.954Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
6219	2024-12-17T21:38:05.954Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
6220	2024-12-17T21:38:05.954Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6221	2024-12-17T21:38:05.954Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
6222	2024-12-17T21:38:05.956Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6223	2024-12-17T21:38:05.957Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6224	2024-12-17T21:38:05.958Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6225	2024-12-17T21:38:05.959Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
6226	2024-12-17T21:38:05.959Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
6227	2024-12-17T21:38:05.959Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
6228	2024-12-17T21:38:05.959Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6229	2024-12-17T21:38:05.961Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
6230	2024-12-17T21:38:05.963Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
6231	2024-12-17T21:38:05.963Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
6232	2024-12-17T21:38:05.963Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
6233	2024-12-17T21:38:05.963Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
6234	2024-12-17T21:38:05.963Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6235	2024-12-17T21:38:05.965Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
6236	2024-12-17T21:38:05.965Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
6237	2024-12-17T21:38:05.965Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
6238	2024-12-17T21:38:05.968Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6239	2024-12-17T21:38:05.968Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6240	2024-12-17T21:38:05.969Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6241	2024-12-17T21:38:05.969Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6242	2024-12-17T21:38:05.969Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6243	2024-12-17T21:38:05.969Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6244	2024-12-17T21:38:05.969Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6245	2024-12-17T21:38:05.969Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6246	2024-12-17T21:38:05.970Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6247	2024-12-17T21:38:05.971Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6248	2024-12-17T21:38:05.971Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-d3278cd4-2587-4f0f-89dd-b4d4669a6ffe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2144, tv_nsec: 106640042 } })
6249	2024-12-17T21:38:05.972Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
6250	2024-12-17T21:38:05.973Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
6251	2024-12-17T21:38:05.973Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
6252	2024-12-17T21:38:05.974Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
6253	2024-12-17T21:38:05.974Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6254	2024-12-17T21:38:05.974Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
6255	2024-12-17T21:38:05.974Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6256	2024-12-17T21:38:05.974Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6257	2024-12-17T21:38:05.974Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6258	2024-12-17T21:38:05.976Z	DEBG	nexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 43.96355ms iteration = 7 status = {"nsuccess":5}
6259	2024-12-17T21:38:05.977Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6260	2024-12-17T21:38:05.977Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6261	2024-12-17T21:38:05.978Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6262	2024-12-17T21:38:05.978Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6263	2024-12-17T21:38:05.978Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 45.432009ms iteration = 31 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
6264	2024-12-17T21:38:05.979Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6265	2024-12-17T21:38:05.980Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
6266	2024-12-17T21:38:05.980Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6267	2024-12-17T21:38:05.980Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6268	2024-12-17T21:38:05.980Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
6269	2024-12-17T21:38:05.980Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6270	2024-12-17T21:38:05.980Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6271	2024-12-17T21:38:05.982Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
6272	2024-12-17T21:38:05.982Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6273	2024-12-17T21:38:05.982Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6274	2024-12-17T21:38:05.982Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6275	2024-12-17T21:38:05.982Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
6276	2024-12-17T21:38:05.982Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6277	2024-12-17T21:38:05.982Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6278	2024-12-17T21:38:05.982Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
6279	2024-12-17T21:38:05.982Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6280	2024-12-17T21:38:05.985Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6281	2024-12-17T21:38:05.985Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
6282	2024-12-17T21:38:05.985Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6283	2024-12-17T21:38:05.985Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6284	2024-12-17T21:38:05.985Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6285	2024-12-17T21:38:05.985Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6286	2024-12-17T21:38:05.985Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6287	2024-12-17T21:38:05.986Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6288	2024-12-17T21:38:05.987Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6289	2024-12-17T21:38:05.992Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
6290	2024-12-17T21:38:05.992Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6291	2024-12-17T21:38:05.992Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6292	2024-12-17T21:38:05.992Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6293	2024-12-17T21:38:05.992Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6294	2024-12-17T21:38:05.994Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6295	2024-12-17T21:38:05.994Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6296	2024-12-17T21:38:05.995Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 62.391099ms iteration = 31 status = {"errors":[],"garbage_collect_requested":[]}
6297	2024-12-17T21:38:06.010Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6298	2024-12-17T21:38:06.010Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
6299	2024-12-17T21:38:06.010Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 77.458936ms iteration = 91 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
6300	2024-12-17T21:38:06.010Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6301	2024-12-17T21:38:06.014Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6302	2024-12-17T21:38:06.014Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6303	2024-12-17T21:38:06.016Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
6304	2024-12-17T21:38:06.016Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
6305	2024-12-17T21:38:06.016Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
6306	2024-12-17T21:38:06.016Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6307	2024-12-17T21:38:06.016Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6308	2024-12-17T21:38:06.017Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6309	2024-12-17T21:38:06.018Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6310	2024-12-17T21:38:06.018Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6311	2024-12-17T21:38:06.020Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6312	2024-12-17T21:38:06.021Z	DEBG	nexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
6313	2024-12-17T21:38:06.021Z	INFO	nexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
6314	2024-12-17T21:38:06.021Z	DEBG	nexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 88.072654ms iteration = 16 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
6315	2024-12-17T21:38:06.022Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
6316	2024-12-17T21:38:06.022Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
6317	2024-12-17T21:38:06.022Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
6318	2024-12-17T21:38:06.022Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6319	2024-12-17T21:38:06.023Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: d3278cd4-2587-4f0f-89dd-b4d4669a6ffe, lookup_type: ById(d3278cd4-2587-4f0f-89dd-b4d4669a6ffe) } result = Ok(())
6320	2024-12-17T21:38:06.023Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6321	2024-12-17T21:38:06.023Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6322	2024-12-17T21:38:06.023Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6323	2024-12-17T21:38:06.023Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
6324	2024-12-17T21:38:06.023Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6325	2024-12-17T21:38:06.023Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6326	2024-12-17T21:38:06.024Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6327	2024-12-17T21:38:06.026Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 93.274554ms iteration = 31 status = 0
6328	2024-12-17T21:38:06.026Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
6329	2024-12-17T21:38:06.026Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 93.289675ms iteration = 31 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
6330	2024-12-17T21:38:06.029Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
6331	2024-12-17T21:38:06.029Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6332	2024-12-17T21:38:06.029Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6333	2024-12-17T21:38:06.033Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6334	2024-12-17T21:38:06.033Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6335	2024-12-17T21:38:06.034Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 100.732977ms iteration = 31 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
6336	2024-12-17T21:38:06.034Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6337	2024-12-17T21:38:06.034Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
6338	2024-12-17T21:38:06.034Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6339	2024-12-17T21:38:06.034Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6340	2024-12-17T21:38:06.034Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6341	2024-12-17T21:38:06.036Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6342	2024-12-17T21:38:06.036Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
6343	2024-12-17T21:38:06.036Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6344	2024-12-17T21:38:06.036Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6345	2024-12-17T21:38:06.036Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6346	2024-12-17T21:38:06.040Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6347	2024-12-17T21:38:06.040Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6348	2024-12-17T21:38:06.042Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6349	2024-12-17T21:38:06.045Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 111.788235ms iteration = 91 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6350	2024-12-17T21:38:06.045Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
6351	2024-12-17T21:38:06.045Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6352	2024-12-17T21:38:06.045Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6353	2024-12-17T21:38:06.048Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6354	2024-12-17T21:38:06.048Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6355	2024-12-17T21:38:06.050Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6356	2024-12-17T21:38:06.050Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6357	2024-12-17T21:38:06.052Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6358	2024-12-17T21:38:06.052Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6359	2024-12-17T21:38:06.055Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6360	2024-12-17T21:38:06.055Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6361	2024-12-17T21:38:06.057Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6362	2024-12-17T21:38:06.057Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6363	2024-12-17T21:38:06.058Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6364	2024-12-17T21:38:06.059Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6365	2024-12-17T21:38:06.059Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
6366	2024-12-17T21:38:06.059Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
6367	2024-12-17T21:38:06.060Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6368	2024-12-17T21:38:06.063Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
6369	2024-12-17T21:38:06.063Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6370	2024-12-17T21:38:06.063Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6371	2024-12-17T21:38:06.065Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6372	2024-12-17T21:38:06.065Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6373	2024-12-17T21:38:06.065Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
6374	2024-12-17T21:38:06.065Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 574f0c35-b8f2-48fe-831c-78afc4a62aaa dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
6375	2024-12-17T21:38:06.065Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6376	2024-12-17T21:38:06.065Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6377	2024-12-17T21:38:06.067Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6378	2024-12-17T21:38:06.067Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6379	2024-12-17T21:38:06.067Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6380	2024-12-17T21:38:06.068Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
6381	2024-12-17T21:38:06.068Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
6382	2024-12-17T21:38:06.069Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 136.585426ms iteration = 31 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
6383	2024-12-17T21:38:06.070Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
6384	2024-12-17T21:38:06.070Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6385	2024-12-17T21:38:06.070Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6386	2024-12-17T21:38:06.070Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6387	2024-12-17T21:38:06.070Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6388	2024-12-17T21:38:06.073Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6389	2024-12-17T21:38:06.073Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 574f0c35-b8f2-48fe-831c-78afc4a62aaa dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
6390	2024-12-17T21:38:06.073Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
6391	2024-12-17T21:38:06.073Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-17 21:23:48.446255 UTC dns_group = internal generation = 1
6392	2024-12-17T21:38:06.073Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 140.177223ms iteration = 16 status = {"generation":1}
6393	2024-12-17T21:38:06.073Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6394	2024-12-17T21:38:06.074Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6395	2024-12-17T21:38:06.075Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6396	2024-12-17T21:38:06.076Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6397	2024-12-17T21:38:06.076Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6398	2024-12-17T21:38:06.077Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6399	2024-12-17T21:38:06.077Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
6400	2024-12-17T21:38:06.077Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 144.527501ms iteration = 31 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
6401	2024-12-17T21:38:06.078Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6402	2024-12-17T21:38:06.079Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
6403	2024-12-17T21:38:06.079Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = ea6acdde-77d4-4bf6-b578-d9a0a29ac37b dns_zone_name = oxide.test found_so_far = 0 version = 2
6404	2024-12-17T21:38:06.079Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6405	2024-12-17T21:38:06.079Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6406	2024-12-17T21:38:06.080Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6407	2024-12-17T21:38:06.082Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
6408	2024-12-17T21:38:06.082Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6409	2024-12-17T21:38:06.083Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6410	2024-12-17T21:38:06.085Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
6411	2024-12-17T21:38:06.085Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6412	2024-12-17T21:38:06.085Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
6413	2024-12-17T21:38:06.085Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6414	2024-12-17T21:38:06.085Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6415	2024-12-17T21:38:06.085Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2bf6071c-c43e-47e0-932d-3ccb3284ebdb", "content-length": "26", "date": "Tue, 17 Dec 2024 21:38:05 GMT"} })
6416	2024-12-17T21:38:06.085Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6417	2024-12-17T21:38:06.090Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
6418	2024-12-17T21:38:06.091Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
6419	2024-12-17T21:38:06.091Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6420	2024-12-17T21:38:06.091Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5bccb04a-8d1c-4f0d-8756-7cfca50a0894", "content-length": "26", "date": "Tue, 17 Dec 2024 21:38:05 GMT"} })
6421	2024-12-17T21:38:06.091Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6422	2024-12-17T21:38:06.093Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
6423	2024-12-17T21:38:06.102Z	DEBG	nexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-17T21:23:03.821125Z, time_modified: 2024-12-17T21:23:03.821125Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 81e78e1e-e55a-44f9-abdc-c9c916e1d9fd, db_silo: Silo { identity: SiloIdentity { id: 81e78e1e-e55a-44f9-abdc-c9c916e1d9fd, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-17T21:23:49.443930Z, time_modified: 2024-12-17T21:23:49.443930Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "e275643b21ab02733b9616c9cd3b0193e27c7f6294b7d310d6f1726c72768e50" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 81e78e1e-e55a-44f9-abdc-c9c916e1d9fd, db_silo: Silo { identity: SiloIdentity { id: 81e78e1e-e55a-44f9-abdc-c9c916e1d9fd, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-17T21:23:49.443930Z, time_modified: 2024-12-17T21:23:49.443930Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "e275643b21ab02733b9616c9cd3b0193e27c7f6294b7d310d6f1726c72768e50" }] }) }
6424	2024-12-17T21:38:06.102Z	DEBG	nexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 168.922292ms iteration = 16 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"81e78e1e-e55a-44f9-abdc-c9c916e1d9fd","tls_certs":["e275643b21ab02733b9616c9cd3b0193e27c7f6294b7d310d6f1726c72768e50"]}},"default_endpoint":{"silo_id":"81e78e1e-e55a-44f9-abdc-c9c916e1d9fd","tls_certs":["e275643b21ab02733b9616c9cd3b0193e27c7f6294b7d310d6f1726c72768e50"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
6425	2024-12-17T21:38:06.116Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 183.030004ms iteration = 33 status = {"physical_disks_added":0}
6426	2024-12-17T21:38:06.119Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6427	2024-12-17T21:38:06.119Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6428	2024-12-17T21:38:06.150Z	WARN	nexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
6429	2024-12-17T21:38:06.150Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 216.689225ms iteration = 16 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
6430	2024-12-17T21:38:06.196Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 zone_address = fd00:1122:3344:101::2
6431	2024-12-17T21:38:06.196Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6432	2024-12-17T21:38:06.197Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb7a9c6d-b271-4fcd-b7a5-c152bf2627bf", "content-length": "26", "date": "Tue, 17 Dec 2024 21:38:05 GMT"} })
6433	2024-12-17T21:38:06.197Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6434	2024-12-17T21:38:06.198Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
6435	2024-12-17T21:38:06.226Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
6436	2024-12-17T21:38:06.258Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
6437	2024-12-17T21:38:06.258Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "980b3d81-5392-4265-a210-cdef0135814a", "content-length": "2", "date": "Tue, 17 Dec 2024 21:38:05 GMT"} })
6438	2024-12-17T21:38:06.260Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
6439	2024-12-17T21:38:06.261Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6440	2024-12-17T21:38:06.270Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 337.423212ms iteration = 31 status = {}
6441	2024-12-17T21:38:06.286Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
6442	2024-12-17T21:38:06.286Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "1dacc348-44f0-450d-81fa-f39703d60085", "date": "Tue, 17 Dec 2024 21:38:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
6443	2024-12-17T21:38:06.286Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
6444	2024-12-17T21:38:06.289Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "1efb61c6-baa0-491f-bfcf-aed13d2a46d8", "date": "Tue, 17 Dec 2024 21:38:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
6445	2024-12-17T21:38:06.289Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
6446	2024-12-17T21:38:06.289Z	INFO	nexus (BackgroundTasks): VPC route manager sled d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
6447	2024-12-17T21:38:06.289Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
6448	2024-12-17T21:38:06.289Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "383af301-b5f2-4a7a-a3de-5652203e354b", "content-length": "372", "date": "Tue, 17 Dec 2024 21:38:06 GMT"} })
6449	2024-12-17T21:38:06.289Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6450	2024-12-17T21:38:06.289Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6451	2024-12-17T21:38:06.290Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
6452	2024-12-17T21:38:06.290Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 357.68887ms iteration = 18 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
6453	2024-12-17T21:38:06.290Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6454	2024-12-17T21:38:06.291Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6455	2024-12-17T21:38:06.291Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6456	2024-12-17T21:38:06.292Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = ea6acdde-77d4-4bf6-b578-d9a0a29ac37b dns_zone_name = oxide.test found_so_far = 1 version = 2
6457	2024-12-17T21:38:06.292Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
6458	2024-12-17T21:38:06.292Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-17 21:23:49.491999 UTC dns_group = external generation = 2
6459	2024-12-17T21:38:06.292Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 359.120529ms iteration = 16 status = {"generation":2}
6460	2024-12-17T21:38:06.293Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 360.080935ms iteration = 31 status = null
6461	2024-12-17T21:38:06.293Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6462	2024-12-17T21:38:06.293Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6463	2024-12-17T21:38:06.293Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
6464	2024-12-17T21:38:06.293Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6465	2024-12-17T21:38:06.294Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6466	2024-12-17T21:38:06.294Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6467	2024-12-17T21:38:06.295Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6468	2024-12-17T21:38:06.297Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
6469	2024-12-17T21:38:06.297Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6470	2024-12-17T21:38:06.297Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6471	2024-12-17T21:38:06.298Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
6472	2024-12-17T21:38:06.298Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6473	2024-12-17T21:38:06.298Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6474	2024-12-17T21:38:06.303Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6475	2024-12-17T21:38:06.303Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6476	2024-12-17T21:38:06.305Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6477	2024-12-17T21:38:06.305Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6478	2024-12-17T21:38:06.306Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6479	2024-12-17T21:38:06.307Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6480	2024-12-17T21:38:06.309Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6481	2024-12-17T21:38:06.309Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6482	2024-12-17T21:38:06.331Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
6483	2024-12-17T21:38:06.331Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
6484	2024-12-17T21:38:06.332Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "541d44cf-2368-4b32-9485-420a2d283ea1", "date": "Tue, 17 Dec 2024 21:38:06 GMT"} })
6485	2024-12-17T21:38:06.332Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6486	2024-12-17T21:38:06.332Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
6487	2024-12-17T21:38:06.332Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6488	2024-12-17T21:38:06.332Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "aba71fc5-05fb-404a-894d-b58f26b3ff7b", "date": "Tue, 17 Dec 2024 21:38:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
6489	2024-12-17T21:38:06.332Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
6490	2024-12-17T21:38:06.334Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "f1c1b537-a3b8-470c-bb95-ca7bf8c926b1", "date": "Tue, 17 Dec 2024 21:38:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
6491	2024-12-17T21:38:06.334Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
6492	2024-12-17T21:38:06.336Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "8ed734b0-9a43-4937-b361-b02ccd7fc9db", "date": "Tue, 17 Dec 2024 21:38:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
6493	2024-12-17T21:38:06.336Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
6494	2024-12-17T21:38:06.339Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
6495	2024-12-17T21:38:06.339Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 406.314966ms iteration = 18 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
6496	2024-12-17T21:38:06.353Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
6497	2024-12-17T21:38:06.353Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "75708522-3684-4af5-bf92-38abcf122348", "content-length": "691", "date": "Tue, 17 Dec 2024 21:38:06 GMT"} })
6498	2024-12-17T21:38:06.353Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
6499	2024-12-17T21:38:06.353Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
6500	2024-12-17T21:38:06.353Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
6501	2024-12-17T21:38:06.355Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6502	2024-12-17T21:38:06.355Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 422.592527ms iteration = 31 status = {}
6503	2024-12-17T21:38:06.355Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6504	2024-12-17T21:38:06.355Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6505	2024-12-17T21:38:06.356Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6506	2024-12-17T21:38:06.357Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6507	2024-12-17T21:38:06.357Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6508	2024-12-17T21:38:06.358Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6509	2024-12-17T21:38:06.358Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6510	2024-12-17T21:38:06.359Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6511	2024-12-17T21:38:06.359Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6512	2024-12-17T21:38:06.360Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6513	2024-12-17T21:38:06.362Z	INFO	nexus (BackgroundTasks): VPC route manager sled d3278cd4-2587-4f0f-89dd-b4d4669a6ffe push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
6514	2024-12-17T21:38:06.362Z	DEBG	nexus (BackgroundTasks): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
6515	2024-12-17T21:38:06.363Z	DEBG	nexus (BackgroundTasks): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "6fd4fd7b-c275-4159-a779-b1b0b91c6da1", "date": "Tue, 17 Dec 2024 21:38:06 GMT"} })
6516	2024-12-17T21:38:06.364Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 431.306503ms iteration = 31 status = {}
6517	2024-12-17T21:38:06.364Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
6518	2024-12-17T21:38:06.364Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6519	2024-12-17T21:38:06.364Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6520	2024-12-17T21:38:06.366Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6521	2024-12-17T21:38:06.366Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6522	2024-12-17T21:38:06.368Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6523	2024-12-17T21:38:06.368Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6524	2024-12-17T21:38:06.369Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6525	2024-12-17T21:38:06.372Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
6526	2024-12-17T21:38:06.372Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6527	2024-12-17T21:38:06.373Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6528	2024-12-17T21:38:06.380Z	DEBG	nexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 1950ee03-4e2e-4a6c-a8f2-88cfbaaaf516, kind: Service { id: 2fd5ca9b-ccee-4a78-97bd-4bd196880cfb }, name: Name("nexus-2fd5ca9b-ccee-4a78-97bd-4bd196880cfb"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 213, 8])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 51cf3c6b-ab6e-43a2-a08f-c7929f83346c, kind: Service { id: 0536d936-3647-4bc5-9d28-7041dd1a1d35 }, name: Name("nexus-0536d936-3647-4bc5-9d28-7041dd1a1d35"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f52af8c0-2e1a-4692-9d7e-5972fdf664ce, kind: Service { id: be092aa0-9f0f-4665-8dd8-c2499ee65aff }, name: Name("nexus-be092aa0-9f0f-4665-8dd8-c2499ee65aff"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 196, 114])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 90400ecd-43ab-4b42-bc2c-68e9b0a8cb02, kind: Service { id: 5dff2793-3ce8-4251-ba71-1a446966cbf7 }, name: Name("external-dns-5dff2793-3ce8-4251-ba71-1a446966cbf7"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 201, 90])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 907c8800-a1ea-47ff-bdbf-63402c8c3e67, kind: Service { id: a761a47e-08b9-41c4-9bd8-8d77651dfa62 }, name: Name("external-dns-a761a47e-08b9-41c4-9bd8-8d77651dfa62"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 253, 16])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
6529	2024-12-17T21:38:06.380Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6530	2024-12-17T21:38:06.380Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6531	2024-12-17T21:38:06.382Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6532	2024-12-17T21:38:06.383Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
6533	2024-12-17T21:38:06.383Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6534	2024-12-17T21:38:06.383Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6535	2024-12-17T21:38:06.385Z	DEBG	nexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
6536	2024-12-17T21:38:06.385Z	DEBG	nexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 90400ecd-43ab-4b42-bc2c-68e9b0a8cb02, kind: Service { id: 5dff2793-3ce8-4251-ba71-1a446966cbf7 }, name: Name("external-dns-5dff2793-3ce8-4251-ba71-1a446966cbf7"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 201, 90])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 907c8800-a1ea-47ff-bdbf-63402c8c3e67, kind: Service { id: a761a47e-08b9-41c4-9bd8-8d77651dfa62 }, name: Name("external-dns-a761a47e-08b9-41c4-9bd8-8d77651dfa62"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 253, 16])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 1950ee03-4e2e-4a6c-a8f2-88cfbaaaf516, kind: Service { id: 2fd5ca9b-ccee-4a78-97bd-4bd196880cfb }, name: Name("nexus-2fd5ca9b-ccee-4a78-97bd-4bd196880cfb"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 213, 8])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 51cf3c6b-ab6e-43a2-a08f-c7929f83346c, kind: Service { id: 0536d936-3647-4bc5-9d28-7041dd1a1d35 }, name: Name("nexus-0536d936-3647-4bc5-9d28-7041dd1a1d35"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f52af8c0-2e1a-4692-9d7e-5972fdf664ce, kind: Service { id: be092aa0-9f0f-4665-8dd8-c2499ee65aff }, name: Name("nexus-be092aa0-9f0f-4665-8dd8-c2499ee65aff"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 196, 114])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
6537	2024-12-17T21:38:06.385Z	DEBG	nexus (service-firewall-rule-progator): resolved 2 rules for sleds background_task = service_firewall_rule_propagation
6538	2024-12-17T21:38:06.400Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6539	2024-12-17T21:38:06.401Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6540	2024-12-17T21:38:06.406Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 1a3e1a79-c78b-4fb4-921b-c5e1e32e8b40, rack_id: 8216c8d1-8011-434d-a57a-2b1ce2e89ad8, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(d6c2617a-6b89-4779-bc70-3002fb0a7e5e) } rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0
6541	2024-12-17T21:38:06.406Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6542	2024-12-17T21:38:06.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6543	2024-12-17T21:38:06.417Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
6544	2024-12-17T21:38:06.417Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee9418f1-ba87-46b9-ba4d-b1e387ff7e0c", "content-length": "134", "date": "Tue, 17 Dec 2024 21:38:05 GMT"} })
6545	2024-12-17T21:38:06.417Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
6546	2024-12-17T21:38:06.417Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = d6c2617a-6b89-4779-bc70-3002fb0a7e5e
6547	2024-12-17T21:38:06.417Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
6548	2024-12-17T21:38:06.417Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
6549	2024-12-17T21:38:06.417Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
6550	2024-12-17T21:38:06.417Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
6551	2024-12-17T21:38:06.417Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
6552	2024-12-17T21:38:06.417Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
6553	2024-12-17T21:38:06.417Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
6554	2024-12-17T21:38:06.417Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
6555	2024-12-17T21:38:06.418Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8
6556	2024-12-17T21:38:06.418Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6557	2024-12-17T21:38:06.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6558	2024-12-17T21:38:06.420Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
6559	2024-12-17T21:38:06.420Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "3a59785f-eb35-4710-ae3a-86fb7902c1dd", "content-length": "2", "date": "Tue, 17 Dec 2024 21:38:05 GMT"} })
6560	2024-12-17T21:38:06.420Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
6561	2024-12-17T21:38:06.420Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c42a82c-e5d0-4ec6-9ede-abc88c7728fe", "content-length": "51", "date": "Tue, 17 Dec 2024 21:38:05 GMT"} })
6562	2024-12-17T21:38:06.420Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch_location = Switch0
6563	2024-12-17T21:38:06.420Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
6564	2024-12-17T21:38:06.496Z	DEBG	nexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: d3278cd4-2587-4f0f-89dd-b4d4669a6ffe, time_created: 2024-12-17T21:23:45.515540Z, time_modified: 2024-12-17T21:23:45.515540Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 8216c8d1-8011-434d-a57a-2b1ce2e89ad8, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(51474595840)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::1:0, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)) }]
6565	2024-12-17T21:38:06.496Z	DEBG	nexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
6566	2024-12-17T21:38:06.496Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6567	2024-12-17T21:38:06.496Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6568	2024-12-17T21:38:06.498Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6569	2024-12-17T21:38:06.498Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6570	2024-12-17T21:38:06.499Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6571	2024-12-17T21:38:06.502Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: d3278cd4-2587-4f0f-89dd-b4d4669a6ffe, lookup_type: ById(d3278cd4-2587-4f0f-89dd-b4d4669a6ffe) } result = Ok(())
6572	2024-12-17T21:38:06.534Z	DEBG	nexus (service-firewall-rule-progator): client request SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
6573	2024-12-17T21:38:06.535Z	DEBG	nexus (service-firewall-rule-progator): client response SledAgent = d3278cd4-2587-4f0f-89dd-b4d4669a6ffe background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "858e16b1-a562-42da-9878-89661a3f5592", "date": "Tue, 17 Dec 2024 21:38:06 GMT"} })
6574	2024-12-17T21:38:06.536Z	INFO	nexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:496
6575	2024-12-17T21:38:06.536Z	DEBG	nexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 602.902216ms
6576	2024-12-17T21:38:06.536Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 602.912348ms iteration = 4 status = {}
6577	2024-12-17T21:38:06.587Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "1f86c6ad-aac4-40bd-a542-c6f59af65fca", "date": "Tue, 17 Dec 2024 21:38:06 GMT"} })
6578	2024-12-17T21:38:06.587Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 switch = Switch0
6579	2024-12-17T21:38:06.587Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
6580	2024-12-17T21:38:06.587Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 8216c8d1-8011-434d-a57a-2b1ce2e89ad8 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "f90b0634-23a3-434c-86c1-b68261737efb", "content-length": "682", "date": "Tue, 17 Dec 2024 21:38:06 GMT"} })
6581	2024-12-17T21:38:06.587Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6582	2024-12-17T21:38:06.587Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6583	2024-12-17T21:38:06.591Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6584	2024-12-17T21:38:06.591Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6585	2024-12-17T21:38:06.593Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6586	2024-12-17T21:38:06.594Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6587	2024-12-17T21:38:06.595Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6588	2024-12-17T21:38:06.595Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6589	2024-12-17T21:38:06.603Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 670.695632ms iteration = 31 status = {}
6590	2024-12-17T21:38:10.392Z	INFO	nexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:49027
6591	2024-12-17T21:38:10.392Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:49027 req_id = 2cdf017d-e32d-4837-a1d7-92d79ef09015 roles = RoleSet { roles: {} } uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?sort_by=id_ascending
6592	2024-12-17T21:38:10.392Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:49027 req_id = 2cdf017d-e32d-4837-a1d7-92d79ef09015 resource = Database result = Ok(()) uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?sort_by=id_ascending
6593	2024-12-17T21:38:10.394Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2430 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:49027 req_id = 2cdf017d-e32d-4837-a1d7-92d79ef09015 response_code = 200 uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?sort_by=id_ascending
6594	2024-12-17T21:38:10.394Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:49027 req_id = 5eebb567-2d9f-4961-b98c-a9ffc29de86d roles = RoleSet { roles: {} } uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMTkzZjYyMi00MGY1LTQyOGYtYmVhMy01MmNjNjdlNzhhMzkifX0%3D
6595	2024-12-17T21:38:10.395Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:49027 req_id = 5eebb567-2d9f-4961-b98c-a9ffc29de86d resource = Database result = Ok(()) uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMTkzZjYyMi00MGY1LTQyOGYtYmVhMy01MmNjNjdlNzhhMzkifX0%3D
6596	2024-12-17T21:38:10.397Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2607 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:49027 req_id = 5eebb567-2d9f-4961-b98c-a9ffc29de86d response_code = 200 uri = /metrics/collectors/952238fd-7df7-4f2b-b8ee-8be2d7488464/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMTkzZjYyMi00MGY1LTQyOGYtYmVhMy01MmNjNjdlNzhhMzkifX0%3D
6597	2024-12-17T21:38:10.403Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 293 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = 823df0b5-8e6d-4be1-8bfc-3ded8b3c9365 response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb
6598	2024-12-17T21:38:15.932Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 92 reason = Timeout
6599	2024-12-17T21:38:15.933Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6600	2024-12-17T21:38:15.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6601	2024-12-17T21:38:15.933Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 92 reason = Timeout
6602	2024-12-17T21:38:15.933Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6603	2024-12-17T21:38:15.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6604	2024-12-17T21:38:15.934Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6605	2024-12-17T21:38:15.935Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6606	2024-12-17T21:38:15.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6607	2024-12-17T21:38:15.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6608	2024-12-17T21:38:15.937Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.886235ms iteration = 92 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6609	2024-12-17T21:38:15.937Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6610	2024-12-17T21:38:15.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6611	2024-12-17T21:38:15.939Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6612	2024-12-17T21:38:15.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6613	2024-12-17T21:38:15.941Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6614	2024-12-17T21:38:15.943Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5, lookup_type: ById(ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5) } result = Ok(())
6615	2024-12-17T21:38:15.943Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6616	2024-12-17T21:38:15.943Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6617	2024-12-17T21:38:15.952Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 original_time_created = 2024-12-17 21:22:56.557129 UTC target_id = ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5 time_created = 2024-12-17 21:22:56.557129 UTC
6618	2024-12-17T21:38:15.952Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.386415ms iteration = 92 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"ac2f5e17-3ba8-443f-84b5-8e903c4c1ed5","time_created":"2024-12-17T21:22:56.557129Z"}
6619	2024-12-17T21:38:20.404Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 427 local_addr = [fd00:1122:3344:101::c]:57610 method = GET remote_addr = [fd00:1122:3344:101::d]:53038 req_id = 18ba19a9-f828-4d98-adb1-fdcd28b943f1 response_code = 200 uri = /2fd5ca9b-ccee-4a78-97bd-4bd196880cfb