Max level shown: TRCE DEBG INFO WARN ERRO FATA

1	2024-12-20T23:00:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 13 reason = Timeout
2	2024-12-20T23:00:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
3	2024-12-20T23:00:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 13 reason = Timeout
4	2024-12-20T23:00:02.384Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
5	2024-12-20T23:00:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6	2024-12-20T23:00:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 13 reason = Timeout
7	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 4 reason = Timeout
8	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
9	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
10	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = external_endpoints iteration = 7 reason = Timeout
11	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 13 reason = Timeout
12	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
13	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
14	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 7 reason = Timeout
15	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
16	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
17	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 13 reason = Timeout
18	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
19	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 7 reason = Timeout
20	2024-12-20T23:00:02.385Z	INFO	nexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
21	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
22	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 13 reason = Timeout
23	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 7 reason = Timeout
24	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 14 reason = Timeout
25	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 7 reason = Timeout
26	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
27	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 7 reason = Timeout
28	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
29	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
30	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 9 reason = Timeout
31	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 13 reason = Timeout
32	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
33	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 8 reason = Timeout
34	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 13 reason = Timeout
35	2024-12-20T23:00:02.385Z	WARN	nexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7
36	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 7 reason = Timeout
37	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
38	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 14.681\u{b5}s iteration = 8 status = {"enabled":false,"target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7"}
39	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 13 reason = Timeout
40	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 13 reason = Timeout
41	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 7 reason = Timeout
42	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
43	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
44	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 7 reason = Timeout
45	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
46	2024-12-20T23:00:02.385Z	INFO	nexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-20 22:50:02.385198125 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
47	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
48	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 13 reason = Timeout
49	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 7 reason = Timeout
51	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
52	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
53	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 37 reason = Timeout
54	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
55	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 37 reason = Timeout
56	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
57	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 13 reason = Timeout
58	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 13 reason = Timeout
59	2024-12-20T23:00:02.385Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
60	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
61	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 13 reason = Timeout
62	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
63	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
64	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
65	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 9 reason = Timeout
66	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
67	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
68	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
69	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
70	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
71	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
72	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
73	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
74	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
75	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
76	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
77	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
78	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
79	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
80	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
81	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
82	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
83	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
84	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
85	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
86	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
87	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
88	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
89	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
90	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
91	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
92	2024-12-20T23:00:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
93	2024-12-20T23:00:02.385Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("616ce87c-1f39-49d9-ba35-ee63b2650ecd.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b4615881-2f68-4c82-be17-071bf3ae3641.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c9217e92-0af3-4930-acb2-aa82ded5b782.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("c9217e92-0af3-4930-acb2-aa82ded5b782.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 1017, tv_nsec: 775666383 } })
94	2024-12-20T23:00:02.385Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1e296b8d-6800-4056-8034-a1ec2d005fff.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("ace0592c-7755-46f8-b47a-c9b6444fe64c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("ace0592c-7755-46f8-b47a-c9b6444fe64c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1017, tv_nsec: 775686084 } })
95	2024-12-20T23:00:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
96	2024-12-20T23:00:02.386Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
97	2024-12-20T23:00:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
98	2024-12-20T23:00:02.386Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
99	2024-12-20T23:00:02.386Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
100	2024-12-20T23:00:02.386Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.417529ms iteration = 7 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
101	2024-12-20T23:00:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
102	2024-12-20T23:00:02.386Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
103	2024-12-20T23:00:02.386Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.504322ms iteration = 7 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
104	2024-12-20T23:00:02.388Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
105	2024-12-20T23:00:02.388Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
106	2024-12-20T23:00:02.388Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1017, tv_nsec: 778550232 } })
107	2024-12-20T23:00:02.389Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
108	2024-12-20T23:00:02.390Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 5.298145ms iteration = 13 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
109	2024-12-20T23:00:02.392Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110	2024-12-20T23:00:02.393Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111	2024-12-20T23:00:02.395Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
112	2024-12-20T23:00:02.395Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
113	2024-12-20T23:00:02.395Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
114	2024-12-20T23:00:02.395Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
115	2024-12-20T23:00:02.396Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
116	2024-12-20T23:00:02.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
117	2024-12-20T23:00:02.407Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
118	2024-12-20T23:00:02.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
119	2024-12-20T23:00:02.408Z	INFO	nexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-20 22:50:02.385198125 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
120	2024-12-20T23:00:02.408Z	DEBG	nexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 23.447044ms iteration = 7 status = {"expiration":"2024-12-20T22:50:02.385198125Z","pruned":[]}
121	2024-12-20T23:00:02.415Z	DEBG	nexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 29.966333ms iteration = 7 status = {"errors":[],"found_port_ok":[]}
122	2024-12-20T23:00:02.417Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123	2024-12-20T23:00:02.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
124	2024-12-20T23:00:02.421Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
125	2024-12-20T23:00:02.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
126	2024-12-20T23:00:02.431Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1017, tv_nsec: 821404659 } })
127	2024-12-20T23:00:02.432Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
128	2024-12-20T23:00:02.440Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129	2024-12-20T23:00:02.443Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
130	2024-12-20T23:00:02.443Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
131	2024-12-20T23:00:02.443Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
132	2024-12-20T23:00:02.446Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133	2024-12-20T23:00:02.448Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
134	2024-12-20T23:00:02.448Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
135	2024-12-20T23:00:02.449Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
136	2024-12-20T23:00:02.452Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137	2024-12-20T23:00:02.455Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
138	2024-12-20T23:00:02.455Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
139	2024-12-20T23:00:02.455Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
140	2024-12-20T23:00:02.457Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 72.251584ms iteration = 13 status = {"errors":[],"finish_invoked_ok":[]}
141	2024-12-20T23:00:02.458Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142	2024-12-20T23:00:02.460Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
143	2024-12-20T23:00:02.461Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144	2024-12-20T23:00:02.461Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
145	2024-12-20T23:00:02.461Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
146	2024-12-20T23:00:02.462Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
147	2024-12-20T23:00:02.462Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
148	2024-12-20T23:00:02.463Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149	2024-12-20T23:00:02.464Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
150	2024-12-20T23:00:02.464Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
151	2024-12-20T23:00:02.464Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
152	2024-12-20T23:00:02.464Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
153	2024-12-20T23:00:02.464Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
154	2024-12-20T23:00:02.466Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155	2024-12-20T23:00:02.466Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156	2024-12-20T23:00:02.466Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157	2024-12-20T23:00:02.466Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 81.717354ms iteration = 13 status = {"errors":[],"garbage_collect_requested":[]}
158	2024-12-20T23:00:02.468Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
159	2024-12-20T23:00:02.468Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
160	2024-12-20T23:00:02.468Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
161	2024-12-20T23:00:02.468Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
162	2024-12-20T23:00:02.468Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
163	2024-12-20T23:00:02.469Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
164	2024-12-20T23:00:02.469Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
165	2024-12-20T23:00:02.470Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
166	2024-12-20T23:00:02.470Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
167	2024-12-20T23:00:02.470Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
168	2024-12-20T23:00:02.470Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
169	2024-12-20T23:00:02.470Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
170	2024-12-20T23:00:02.470Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
171	2024-12-20T23:00:02.470Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
172	2024-12-20T23:00:02.470Z	DEBG	nexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
173	2024-12-20T23:00:02.470Z	INFO	nexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
174	2024-12-20T23:00:02.470Z	DEBG	nexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 85.445656ms iteration = 7 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
175	2024-12-20T23:00:02.470Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
176	2024-12-20T23:00:02.471Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
177	2024-12-20T23:00:02.471Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
178	2024-12-20T23:00:02.471Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
179	2024-12-20T23:00:02.473Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
180	2024-12-20T23:00:02.473Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
181	2024-12-20T23:00:02.474Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
182	2024-12-20T23:00:02.474Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
183	2024-12-20T23:00:02.475Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
184	2024-12-20T23:00:02.475Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
185	2024-12-20T23:00:02.475Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
186	2024-12-20T23:00:02.475Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
187	2024-12-20T23:00:02.475Z	DEBG	nexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 90.749952ms iteration = 4 status = {"nsuccess":5}
188	2024-12-20T23:00:02.476Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
189	2024-12-20T23:00:02.477Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
190	2024-12-20T23:00:02.477Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
191	2024-12-20T23:00:02.478Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
192	2024-12-20T23:00:02.479Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
193	2024-12-20T23:00:02.479Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
194	2024-12-20T23:00:02.479Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
195	2024-12-20T23:00:02.479Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
196	2024-12-20T23:00:02.479Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
197	2024-12-20T23:00:02.479Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
198	2024-12-20T23:00:02.479Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
199	2024-12-20T23:00:02.479Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
200	2024-12-20T23:00:02.481Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
201	2024-12-20T23:00:02.481Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
202	2024-12-20T23:00:02.482Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
203	2024-12-20T23:00:02.482Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
204	2024-12-20T23:00:02.483Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
205	2024-12-20T23:00:02.483Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
206	2024-12-20T23:00:02.484Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
207	2024-12-20T23:00:02.484Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
208	2024-12-20T23:00:02.484Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
209	2024-12-20T23:00:02.484Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1017, tv_nsec: 874511357 } })
210	2024-12-20T23:00:02.485Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
211	2024-12-20T23:00:02.487Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
212	2024-12-20T23:00:02.487Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
213	2024-12-20T23:00:02.487Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
214	2024-12-20T23:00:02.490Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
215	2024-12-20T23:00:02.490Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
216	2024-12-20T23:00:02.492Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217	2024-12-20T23:00:02.496Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
218	2024-12-20T23:00:02.496Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
219	2024-12-20T23:00:02.497Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
220	2024-12-20T23:00:02.517Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
221	2024-12-20T23:00:02.518Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
222	2024-12-20T23:00:02.529Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
223	2024-12-20T23:00:02.529Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
224	2024-12-20T23:00:02.529Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
225	2024-12-20T23:00:02.529Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
226	2024-12-20T23:00:02.530Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
227	2024-12-20T23:00:02.533Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
228	2024-12-20T23:00:02.533Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 147.87345ms iteration = 37 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
229	2024-12-20T23:00:02.535Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
230	2024-12-20T23:00:02.535Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
231	2024-12-20T23:00:02.535Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
232	2024-12-20T23:00:02.535Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
233	2024-12-20T23:00:02.536Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
234	2024-12-20T23:00:02.537Z	DEBG	nexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
235	2024-12-20T23:00:02.537Z	INFO	nexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
236	2024-12-20T23:00:02.537Z	DEBG	nexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 152.334373ms iteration = 7 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
237	2024-12-20T23:00:02.538Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
238	2024-12-20T23:00:02.539Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
239	2024-12-20T23:00:02.539Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
240	2024-12-20T23:00:02.539Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 154.545173ms iteration = 13 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
241	2024-12-20T23:00:02.542Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
242	2024-12-20T23:00:02.542Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
243	2024-12-20T23:00:02.542Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
244	2024-12-20T23:00:02.545Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 160.003604ms iteration = 13 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
245	2024-12-20T23:00:02.545Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
246	2024-12-20T23:00:02.545Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 1e86bf23-6ffd-4a12-93c7-0061d7d17590 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
247	2024-12-20T23:00:02.545Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
248	2024-12-20T23:00:02.545Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
249	2024-12-20T23:00:02.545Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
250	2024-12-20T23:00:02.546Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
251	2024-12-20T23:00:02.546Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
252	2024-12-20T23:00:02.546Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
253	2024-12-20T23:00:02.547Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
254	2024-12-20T23:00:02.548Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
255	2024-12-20T23:00:02.548Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 163.211231ms iteration = 13 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
256	2024-12-20T23:00:02.548Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
257	2024-12-20T23:00:02.548Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
258	2024-12-20T23:00:02.549Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
259	2024-12-20T23:00:02.551Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 165.781242ms iteration = 37 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
260	2024-12-20T23:00:02.551Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
261	2024-12-20T23:00:02.551Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
262	2024-12-20T23:00:02.551Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
263	2024-12-20T23:00:02.552Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 167.089138ms iteration = 13 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
264	2024-12-20T23:00:02.552Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
265	2024-12-20T23:00:02.552Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
266	2024-12-20T23:00:02.552Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
267	2024-12-20T23:00:02.553Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
268	2024-12-20T23:00:02.553Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
269	2024-12-20T23:00:02.553Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d88410c0-40f2-4311-9bb0-1e612b7748f6", "content-length": "26", "date": "Fri, 20 Dec 2024 23:00:02 GMT"} })
270	2024-12-20T23:00:02.553Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
271	2024-12-20T23:00:02.554Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
272	2024-12-20T23:00:02.555Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
273	2024-12-20T23:00:02.555Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
274	2024-12-20T23:00:02.555Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 170.243195ms iteration = 13 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
275	2024-12-20T23:00:02.556Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 1e86bf23-6ffd-4a12-93c7-0061d7d17590 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
276	2024-12-20T23:00:02.556Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
277	2024-12-20T23:00:02.556Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-20 22:54:38.956071 UTC dns_group = internal generation = 1
278	2024-12-20T23:00:02.556Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 171.421337ms iteration = 7 status = {"generation":1}
279	2024-12-20T23:00:02.556Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
280	2024-12-20T23:00:02.558Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 287d6774-d3e6-4b73-8c77-c90534d4a54d, lookup_type: ById(287d6774-d3e6-4b73-8c77-c90534d4a54d) } result = Ok(())
281	2024-12-20T23:00:02.558Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
282	2024-12-20T23:00:02.558Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
283	2024-12-20T23:00:02.561Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 176.150326ms iteration = 13 status = 0
284	2024-12-20T23:00:02.563Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
285	2024-12-20T23:00:02.573Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
286	2024-12-20T23:00:02.576Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
287	2024-12-20T23:00:02.576Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
288	2024-12-20T23:00:02.576Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
289	2024-12-20T23:00:02.589Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
290	2024-12-20T23:00:02.589Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
291	2024-12-20T23:00:02.594Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
292	2024-12-20T23:00:02.612Z	DEBG	nexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-20T22:53:58.073373Z, time_modified: 2024-12-20T22:53:58.073373Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, db_silo: Silo { identity: SiloIdentity { id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-20T22:54:39.847015Z, time_modified: 2024-12-20T22:54:39.847015Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, db_silo: Silo { identity: SiloIdentity { id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-20T22:54:39.847015Z, time_modified: 2024-12-20T22:54:39.847015Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156" }] }) }
293	2024-12-20T23:00:02.612Z	DEBG	nexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 227.189318ms iteration = 7 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"ee579470-b16b-4af1-ad3a-b3d29f9a2fb2","tls_certs":["ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156"]}},"default_endpoint":{"silo_id":"ee579470-b16b-4af1-ad3a-b3d29f9a2fb2","tls_certs":["ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
294	2024-12-20T23:00:02.615Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
295	2024-12-20T23:00:02.615Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 5fa20c62-75c0-4531-be07-51b87d303d18 dns_zone_name = oxide.test found_so_far = 0 version = 2
296	2024-12-20T23:00:02.616Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
297	2024-12-20T23:00:02.616Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
298	2024-12-20T23:00:02.617Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
299	2024-12-20T23:00:02.620Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
300	2024-12-20T23:00:02.620Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
301	2024-12-20T23:00:02.620Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
302	2024-12-20T23:00:02.623Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 5fa20c62-75c0-4531-be07-51b87d303d18 dns_zone_name = oxide.test found_so_far = 1 version = 2
303	2024-12-20T23:00:02.623Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
304	2024-12-20T23:00:02.623Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-20 22:54:39.899407 UTC dns_group = external generation = 2
305	2024-12-20T23:00:02.623Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 238.212961ms iteration = 7 status = {"generation":2}
306	2024-12-20T23:00:02.630Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 245.211334ms iteration = 14 status = {"physical_disks_added":0}
307	2024-12-20T23:00:02.641Z	WARN	nexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
308	2024-12-20T23:00:02.641Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 256.827152ms iteration = 7 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
309	2024-12-20T23:00:02.703Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
310	2024-12-20T23:00:02.704Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
311	2024-12-20T23:00:02.713Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 zone_address = fd00:1122:3344:101::2
312	2024-12-20T23:00:02.714Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
313	2024-12-20T23:00:02.714Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a9cbb49-2a84-4768-9fc9-c8d9ad6969ed", "content-length": "26", "date": "Fri, 20 Dec 2024 23:00:02 GMT"} })
314	2024-12-20T23:00:02.714Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
315	2024-12-20T23:00:02.716Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
316	2024-12-20T23:00:02.723Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 338.836454ms iteration = 13 status = {}
317	2024-12-20T23:00:02.751Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
318	2024-12-20T23:00:02.751Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "a49f37ae-e902-4730-bb13-a3bee0ad95b9", "date": "Fri, 20 Dec 2024 23:00:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
319	2024-12-20T23:00:02.751Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
320	2024-12-20T23:00:02.752Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
321	2024-12-20T23:00:02.758Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "a9aea439-78ce-438e-99f2-e7fbacb173f6", "date": "Fri, 20 Dec 2024 23:00:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
322	2024-12-20T23:00:02.758Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
323	2024-12-20T23:00:02.759Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
324	2024-12-20T23:00:02.759Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
325	2024-12-20T23:00:02.759Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
326	2024-12-20T23:00:02.762Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
327	2024-12-20T23:00:02.765Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
328	2024-12-20T23:00:02.765Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 379.689636ms iteration = 9 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
329	2024-12-20T23:00:02.765Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
330	2024-12-20T23:00:02.765Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
331	2024-12-20T23:00:02.765Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
332	2024-12-20T23:00:02.765Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
333	2024-12-20T23:00:02.765Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
334	2024-12-20T23:00:02.765Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9385491c-861f-4ec2-81dd-544f8701774e", "content-length": "26", "date": "Fri, 20 Dec 2024 23:00:02 GMT"} })
335	2024-12-20T23:00:02.765Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
336	2024-12-20T23:00:02.767Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
337	2024-12-20T23:00:02.836Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
338	2024-12-20T23:00:02.836Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
339	2024-12-20T23:00:02.863Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
340	2024-12-20T23:00:02.906Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3d1b345-f837-4719-a851-c6ee08ed4f59", "content-length": "372", "date": "Fri, 20 Dec 2024 23:00:02 GMT"} })
341	2024-12-20T23:00:02.906Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
342	2024-12-20T23:00:02.906Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
343	2024-12-20T23:00:02.959Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
344	2024-12-20T23:00:02.959Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
345	2024-12-20T23:00:02.959Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "f360ed91-4ec3-48e2-b8e2-3004d35e9a96", "date": "Fri, 20 Dec 2024 23:00:02 GMT"} })
346	2024-12-20T23:00:02.959Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
347	2024-12-20T23:00:02.960Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
348	2024-12-20T23:00:02.962Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
349	2024-12-20T23:00:02.962Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
350	2024-12-20T23:00:02.966Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
351	2024-12-20T23:00:02.966Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
352	2024-12-20T23:00:02.970Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
353	2024-12-20T23:00:02.970Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
354	2024-12-20T23:00:02.971Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "87ad62f4-0338-4652-9fc9-20899693f093", "date": "Fri, 20 Dec 2024 23:00:02 GMT"} })
355	2024-12-20T23:00:02.973Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 588.477738ms iteration = 13 status = {}
356	2024-12-20T23:00:02.973Z	DEBG	nexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
357	2024-12-20T23:00:02.973Z	DEBG	nexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
358	2024-12-20T23:00:02.990Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
359	2024-12-20T23:00:02.991Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "92ddc4f5-cbc7-45f6-b09d-c1ff676a8ba1", "content-length": "691", "date": "Fri, 20 Dec 2024 23:00:02 GMT"} })
360	2024-12-20T23:00:02.991Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
361	2024-12-20T23:00:02.991Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
362	2024-12-20T23:00:02.991Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
363	2024-12-20T23:00:02.992Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 607.134131ms iteration = 13 status = {}
364	2024-12-20T23:00:03.053Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
365	2024-12-20T23:00:03.053Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "b4786045-14d0-4a3c-8413-a0c9fa7fc93f", "content-length": "1", "date": "Fri, 20 Dec 2024 23:00:02 GMT"} })
366	2024-12-20T23:00:03.056Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
367	2024-12-20T23:00:03.056Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
368	2024-12-20T23:00:03.059Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 673.70535ms iteration = 13 status = null
369	2024-12-20T23:00:03.064Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
370	2024-12-20T23:00:03.064Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "ac977625-290b-4b72-8841-b1abdc7e8e18", "date": "Fri, 20 Dec 2024 23:00:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
371	2024-12-20T23:00:03.064Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
372	2024-12-20T23:00:03.066Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "9f19e64a-1d0b-4002-aea5-39751e9f777e", "date": "Fri, 20 Dec 2024 23:00:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
373	2024-12-20T23:00:03.066Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
374	2024-12-20T23:00:03.067Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "b92510a4-2fd9-4409-b4a4-561ab5532507", "date": "Fri, 20 Dec 2024 23:00:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
375	2024-12-20T23:00:03.068Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
376	2024-12-20T23:00:03.070Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
377	2024-12-20T23:00:03.070Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 684.943718ms iteration = 9 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
378	2024-12-20T23:00:03.072Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
379	2024-12-20T23:00:03.072Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
380	2024-12-20T23:00:03.074Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 03dd12a0-dc2d-46e1-a7f4-d3397b9b7aa7, rack_id: b394d0c6-19a8-4136-9148-af6c47c1f8e0, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(560ca89c-7c0c-4921-8e5c-b57676f695b0) } rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
381	2024-12-20T23:00:03.074Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
382	2024-12-20T23:00:03.074Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
383	2024-12-20T23:00:03.085Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
384	2024-12-20T23:00:03.086Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "e9651584-a3f0-4a16-9331-214061353758", "content-length": "134", "date": "Fri, 20 Dec 2024 23:00:02 GMT"} })
385	2024-12-20T23:00:03.086Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
386	2024-12-20T23:00:03.086Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 560ca89c-7c0c-4921-8e5c-b57676f695b0
387	2024-12-20T23:00:03.086Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
388	2024-12-20T23:00:03.086Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
389	2024-12-20T23:00:03.086Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
390	2024-12-20T23:00:03.086Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
391	2024-12-20T23:00:03.086Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
392	2024-12-20T23:00:03.086Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
393	2024-12-20T23:00:03.086Z	DEBG	nexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
394	2024-12-20T23:00:03.086Z	INFO	nexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:61266
395	2024-12-20T23:00:03.086Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
396	2024-12-20T23:00:03.086Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
397	2024-12-20T23:00:03.086Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:61266 req_id = e7cef2e3-996e-4d49-af20-f60d567e49e5 roles = RoleSet { roles: {} } uri = /metrics/producers
398	2024-12-20T23:00:03.086Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
399	2024-12-20T23:00:03.086Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
400	2024-12-20T23:00:03.087Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:61266 req_id = e7cef2e3-996e-4d49-af20-f60d567e49e5 resource = Database result = Ok(()) uri = /metrics/producers
401	2024-12-20T23:00:03.087Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
402	2024-12-20T23:00:03.090Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
403	2024-12-20T23:00:03.090Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "6bcf29f0-988e-4184-b4af-316271185dc9", "content-length": "2", "date": "Fri, 20 Dec 2024 23:00:02 GMT"} })
404	2024-12-20T23:00:03.090Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
405	2024-12-20T23:00:03.090Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc0f0fcb-972e-4596-9f87-011978b6ff4b", "content-length": "52", "date": "Fri, 20 Dec 2024 23:00:02 GMT"} })
406	2024-12-20T23:00:03.090Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
407	2024-12-20T23:00:03.090Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
408	2024-12-20T23:00:03.149Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 271 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 67282612-11d1-422d-914e-3c2fd2144d52 response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
409	2024-12-20T23:00:03.165Z	INFO	nexus: assigned collector to new producer collector_id = 103fc383-10f6-4327-a1b1-dabac4e1e365 file = nexus/src/app/oximeter.rs:76 producer_id = f3b60bd3-6e34-4140-a755-57f26dbc5b34
410	2024-12-20T23:00:03.165Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 78699 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:61266 req_id = e7cef2e3-996e-4d49-af20-f60d567e49e5 response_code = 201 uri = /metrics/producers
411	2024-12-20T23:00:03.165Z	DEBG	nexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "e7cef2e3-996e-4d49-af20-f60d567e49e5", "content-length": "41", "date": "Fri, 20 Dec 2024 23:00:03 GMT"} })
412	2024-12-20T23:00:03.166Z	DEBG	nexus (producer-registration-task): registered with nexus successfully
413	2024-12-20T23:00:03.166Z	DEBG	nexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
414	2024-12-20T23:00:03.196Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "eb6a511b-f1fc-4c98-a22e-e32174615593", "date": "Fri, 20 Dec 2024 23:00:02 GMT"} })
415	2024-12-20T23:00:03.196Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch = Switch0
416	2024-12-20T23:00:03.196Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
417	2024-12-20T23:00:03.196Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "bed96585-93e2-48f3-ab6d-20fa18c5973d", "content-length": "682", "date": "Fri, 20 Dec 2024 23:00:02 GMT"} })
418	2024-12-20T23:00:03.196Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
419	2024-12-20T23:00:03.197Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
420	2024-12-20T23:00:03.200Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
421	2024-12-20T23:00:03.201Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
422	2024-12-20T23:00:03.204Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
423	2024-12-20T23:00:03.205Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
424	2024-12-20T23:00:03.207Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
425	2024-12-20T23:00:03.207Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
426	2024-12-20T23:00:03.214Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 829.178307ms iteration = 13 status = {}
427	2024-12-20T23:00:12.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 38 reason = Timeout
428	2024-12-20T23:00:12.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 38 reason = Timeout
429	2024-12-20T23:00:12.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
430	2024-12-20T23:00:12.385Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
431	2024-12-20T23:00:12.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
432	2024-12-20T23:00:12.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
433	2024-12-20T23:00:12.387Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
434	2024-12-20T23:00:12.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
435	2024-12-20T23:00:12.390Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
436	2024-12-20T23:00:12.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
437	2024-12-20T23:00:12.392Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
438	2024-12-20T23:00:12.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
439	2024-12-20T23:00:12.419Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
440	2024-12-20T23:00:12.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
441	2024-12-20T23:00:12.420Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 35.428463ms iteration = 38 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
442	2024-12-20T23:00:12.425Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
443	2024-12-20T23:00:12.428Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
444	2024-12-20T23:00:12.428Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
445	2024-12-20T23:00:12.428Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
446	2024-12-20T23:00:12.439Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
447	2024-12-20T23:00:12.440Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 54.476136ms iteration = 38 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
448	2024-12-20T23:00:13.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 311 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 193890ae-815e-483e-abaf-e36deb61879e response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
449	2024-12-20T23:00:22.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 39 reason = Timeout
450	2024-12-20T23:00:22.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 39 reason = Timeout
451	2024-12-20T23:00:22.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
452	2024-12-20T23:00:22.385Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
453	2024-12-20T23:00:22.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
454	2024-12-20T23:00:22.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
455	2024-12-20T23:00:22.387Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
456	2024-12-20T23:00:22.387Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
457	2024-12-20T23:00:22.388Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
458	2024-12-20T23:00:22.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
459	2024-12-20T23:00:22.389Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
460	2024-12-20T23:00:22.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
461	2024-12-20T23:00:22.391Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
462	2024-12-20T23:00:22.391Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
463	2024-12-20T23:00:22.392Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
464	2024-12-20T23:00:22.394Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
465	2024-12-20T23:00:22.394Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
466	2024-12-20T23:00:22.395Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
467	2024-12-20T23:00:22.407Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
468	2024-12-20T23:00:22.407Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.052036ms iteration = 39 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
469	2024-12-20T23:00:22.423Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 38.235299ms iteration = 39 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
470	2024-12-20T23:00:23.149Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 269 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 4055735e-7203-497f-8cea-159ac5b5fa6b response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
471	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 14 reason = Timeout
472	2024-12-20T23:00:32.385Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
473	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 14 reason = Timeout
474	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
475	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 40 reason = Timeout
476	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
477	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
478	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 14 reason = Timeout
479	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 14 reason = Timeout
480	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
481	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
482	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 14 reason = Timeout
483	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 14 reason = Timeout
484	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
485	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
486	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 14 reason = Timeout
487	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
488	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 14 reason = Timeout
489	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
490	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 14 reason = Timeout
491	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
492	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 14 reason = Timeout
493	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 14 reason = Timeout
494	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
495	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 14 reason = Timeout
496	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
497	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 14 reason = Timeout
498	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 15 reason = Timeout
499	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
500	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
501	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
502	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 40 reason = Timeout
503	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 14 reason = Timeout
504	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
505	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
506	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
507	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
508	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
509	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
510	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
511	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
512	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
513	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
514	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
515	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
516	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
517	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
518	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
519	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
520	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
521	2024-12-20T23:00:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
522	2024-12-20T23:00:32.387Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
523	2024-12-20T23:00:32.388Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
524	2024-12-20T23:00:32.389Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
525	2024-12-20T23:00:32.389Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
526	2024-12-20T23:00:32.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
527	2024-12-20T23:00:32.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
528	2024-12-20T23:00:32.390Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
529	2024-12-20T23:00:32.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
530	2024-12-20T23:00:32.390Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1047, tv_nsec: 780502946 } })
531	2024-12-20T23:00:32.391Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
532	2024-12-20T23:00:32.391Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
533	2024-12-20T23:00:32.392Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
534	2024-12-20T23:00:32.392Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 6.975561ms iteration = 14 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
535	2024-12-20T23:00:32.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
536	2024-12-20T23:00:32.393Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
537	2024-12-20T23:00:32.393Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
538	2024-12-20T23:00:32.393Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
539	2024-12-20T23:00:32.394Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
540	2024-12-20T23:00:32.395Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
541	2024-12-20T23:00:32.397Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
542	2024-12-20T23:00:32.397Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
543	2024-12-20T23:00:32.397Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
544	2024-12-20T23:00:32.400Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
545	2024-12-20T23:00:32.400Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
546	2024-12-20T23:00:32.400Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
547	2024-12-20T23:00:32.400Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
548	2024-12-20T23:00:32.401Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
549	2024-12-20T23:00:32.401Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1047, tv_nsec: 791398595 } })
550	2024-12-20T23:00:32.401Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
551	2024-12-20T23:00:32.409Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1047, tv_nsec: 799034495 } })
552	2024-12-20T23:00:32.409Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
553	2024-12-20T23:00:32.409Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
554	2024-12-20T23:00:32.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
555	2024-12-20T23:00:32.409Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
556	2024-12-20T23:00:32.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
557	2024-12-20T23:00:32.411Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
558	2024-12-20T23:00:32.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
559	2024-12-20T23:00:32.413Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
560	2024-12-20T23:00:32.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
561	2024-12-20T23:00:32.431Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
562	2024-12-20T23:00:32.431Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
563	2024-12-20T23:00:32.431Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
564	2024-12-20T23:00:32.431Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
565	2024-12-20T23:00:32.432Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
566	2024-12-20T23:00:32.432Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
567	2024-12-20T23:00:32.433Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 47.933606ms iteration = 14 status = {"errors":[],"finish_invoked_ok":[]}
568	2024-12-20T23:00:32.435Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
569	2024-12-20T23:00:32.435Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 50.108686ms iteration = 14 status = {"errors":[],"garbage_collect_requested":[]}
570	2024-12-20T23:00:32.436Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
571	2024-12-20T23:00:32.438Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
572	2024-12-20T23:00:32.438Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
573	2024-12-20T23:00:32.438Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
574	2024-12-20T23:00:32.438Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
575	2024-12-20T23:00:32.438Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
576	2024-12-20T23:00:32.438Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
577	2024-12-20T23:00:32.440Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
578	2024-12-20T23:00:32.440Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
579	2024-12-20T23:00:32.442Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
580	2024-12-20T23:00:32.443Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
581	2024-12-20T23:00:32.444Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
582	2024-12-20T23:00:32.445Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
583	2024-12-20T23:00:32.446Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 60.858642ms iteration = 40 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
584	2024-12-20T23:00:32.446Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
585	2024-12-20T23:00:32.446Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 61.280593ms iteration = 14 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
586	2024-12-20T23:00:32.447Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 61.832328ms iteration = 14 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
587	2024-12-20T23:00:32.450Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
588	2024-12-20T23:00:32.453Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
589	2024-12-20T23:00:32.453Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
590	2024-12-20T23:00:32.453Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
591	2024-12-20T23:00:32.462Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 77.086146ms iteration = 15 status = {"physical_disks_added":0}
592	2024-12-20T23:00:32.463Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
593	2024-12-20T23:00:32.463Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
594	2024-12-20T23:00:32.463Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5dfa8bfb-dbd4-4acb-ba50-5d6a2dfbb0b9", "content-length": "26", "date": "Fri, 20 Dec 2024 23:00:32 GMT"} })
595	2024-12-20T23:00:32.463Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
596	2024-12-20T23:00:32.465Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
597	2024-12-20T23:00:32.469Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
598	2024-12-20T23:00:32.469Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
599	2024-12-20T23:00:32.471Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
600	2024-12-20T23:00:32.472Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
601	2024-12-20T23:00:32.472Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
602	2024-12-20T23:00:32.472Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
603	2024-12-20T23:00:32.472Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
604	2024-12-20T23:00:32.473Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
605	2024-12-20T23:00:32.473Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
606	2024-12-20T23:00:32.473Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
607	2024-12-20T23:00:32.473Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
608	2024-12-20T23:00:32.473Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 88.002246ms iteration = 40 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
609	2024-12-20T23:00:32.474Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
610	2024-12-20T23:00:32.477Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 287d6774-d3e6-4b73-8c77-c90534d4a54d, lookup_type: ById(287d6774-d3e6-4b73-8c77-c90534d4a54d) } result = Ok(())
611	2024-12-20T23:00:32.477Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
612	2024-12-20T23:00:32.477Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
613	2024-12-20T23:00:32.482Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 96.764317ms iteration = 14 status = 0
614	2024-12-20T23:00:32.482Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 96.662974ms iteration = 14 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
615	2024-12-20T23:00:32.492Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
616	2024-12-20T23:00:32.493Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
617	2024-12-20T23:00:32.496Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 zone_address = fd00:1122:3344:101::2
618	2024-12-20T23:00:32.496Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
619	2024-12-20T23:00:32.497Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f40535b-bbce-4a1c-8ee0-44ba26ac862d", "content-length": "26", "date": "Fri, 20 Dec 2024 23:00:32 GMT"} })
620	2024-12-20T23:00:32.497Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
621	2024-12-20T23:00:32.499Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
622	2024-12-20T23:00:32.505Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
623	2024-12-20T23:00:32.505Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
624	2024-12-20T23:00:32.505Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "26454759-298c-4cb8-8da2-e9a9fec3d5bd", "content-length": "26", "date": "Fri, 20 Dec 2024 23:00:32 GMT"} })
625	2024-12-20T23:00:32.505Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
626	2024-12-20T23:00:32.507Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
627	2024-12-20T23:00:32.516Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
628	2024-12-20T23:00:32.516Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
629	2024-12-20T23:00:32.516Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 130.830812ms iteration = 14 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
630	2024-12-20T23:00:32.519Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 134.422591ms iteration = 14 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
631	2024-12-20T23:00:32.528Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
632	2024-12-20T23:00:32.530Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
633	2024-12-20T23:00:32.530Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
634	2024-12-20T23:00:32.530Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
635	2024-12-20T23:00:32.532Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
636	2024-12-20T23:00:32.533Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
637	2024-12-20T23:00:32.533Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
638	2024-12-20T23:00:32.533Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
639	2024-12-20T23:00:32.591Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 205.884614ms iteration = 14 status = {}
640	2024-12-20T23:00:32.615Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
641	2024-12-20T23:00:32.615Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
642	2024-12-20T23:00:32.615Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "62859f15-5910-49bd-9dfc-32d2905f0b32", "content-length": "372", "date": "Fri, 20 Dec 2024 23:00:32 GMT"} })
643	2024-12-20T23:00:32.615Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
644	2024-12-20T23:00:32.616Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
645	2024-12-20T23:00:32.663Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
646	2024-12-20T23:00:32.663Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
647	2024-12-20T23:00:32.664Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "3265d55d-cf9a-4182-8112-33a223a7398e", "date": "Fri, 20 Dec 2024 23:00:32 GMT"} })
648	2024-12-20T23:00:32.664Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
649	2024-12-20T23:00:32.664Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
650	2024-12-20T23:00:32.747Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
651	2024-12-20T23:00:32.747Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e263252-4dd1-45e9-bcf0-9d4f71a3e268", "content-length": "1", "date": "Fri, 20 Dec 2024 23:00:32 GMT"} })
652	2024-12-20T23:00:32.750Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
653	2024-12-20T23:00:32.750Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
654	2024-12-20T23:00:32.756Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
655	2024-12-20T23:00:32.757Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1d23918-c944-4fe7-95ce-693903f48610", "content-length": "691", "date": "Fri, 20 Dec 2024 23:00:32 GMT"} })
656	2024-12-20T23:00:32.757Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
657	2024-12-20T23:00:32.757Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
658	2024-12-20T23:00:32.757Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
659	2024-12-20T23:00:32.758Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 373.424914ms iteration = 14 status = {}
660	2024-12-20T23:00:32.759Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 373.625779ms iteration = 14 status = null
661	2024-12-20T23:00:32.759Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
662	2024-12-20T23:00:32.760Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
663	2024-12-20T23:00:32.763Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
664	2024-12-20T23:00:32.763Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
665	2024-12-20T23:00:32.766Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
666	2024-12-20T23:00:32.766Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
667	2024-12-20T23:00:32.767Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "eb3d22a2-bbb8-49f7-8fea-4b5874c2a0a2", "date": "Fri, 20 Dec 2024 23:00:32 GMT"} })
668	2024-12-20T23:00:32.769Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 383.85628ms iteration = 14 status = {}
669	2024-12-20T23:00:32.770Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
670	2024-12-20T23:00:32.770Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
671	2024-12-20T23:00:32.772Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 03dd12a0-dc2d-46e1-a7f4-d3397b9b7aa7, rack_id: b394d0c6-19a8-4136-9148-af6c47c1f8e0, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(560ca89c-7c0c-4921-8e5c-b57676f695b0) } rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
672	2024-12-20T23:00:32.772Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
673	2024-12-20T23:00:32.773Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
674	2024-12-20T23:00:32.790Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
675	2024-12-20T23:00:32.790Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "a664e934-4ccd-43f4-b639-0ef701183f91", "content-length": "134", "date": "Fri, 20 Dec 2024 23:00:32 GMT"} })
676	2024-12-20T23:00:32.790Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
677	2024-12-20T23:00:32.790Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 560ca89c-7c0c-4921-8e5c-b57676f695b0
678	2024-12-20T23:00:32.791Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
679	2024-12-20T23:00:32.791Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
680	2024-12-20T23:00:32.791Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
681	2024-12-20T23:00:32.791Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
682	2024-12-20T23:00:32.791Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
683	2024-12-20T23:00:32.791Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
684	2024-12-20T23:00:32.791Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
685	2024-12-20T23:00:32.791Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
686	2024-12-20T23:00:32.791Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
687	2024-12-20T23:00:32.791Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
688	2024-12-20T23:00:32.792Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
689	2024-12-20T23:00:32.794Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
690	2024-12-20T23:00:32.794Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "b4568b56-d1dc-4378-92fb-4ab5f8fa8a2b", "content-length": "2", "date": "Fri, 20 Dec 2024 23:00:32 GMT"} })
691	2024-12-20T23:00:32.794Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
692	2024-12-20T23:00:32.794Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "eca20da3-a842-4658-afcd-aa5695076c73", "content-length": "52", "date": "Fri, 20 Dec 2024 23:00:32 GMT"} })
693	2024-12-20T23:00:32.794Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
694	2024-12-20T23:00:32.794Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
695	2024-12-20T23:00:32.898Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "9438a909-dcc5-498f-9760-d6bfe1386f31", "date": "Fri, 20 Dec 2024 23:00:32 GMT"} })
696	2024-12-20T23:00:32.898Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch = Switch0
697	2024-12-20T23:00:32.898Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
698	2024-12-20T23:00:32.898Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "d11935ae-73cd-4996-bc03-23167a75fcff", "content-length": "682", "date": "Fri, 20 Dec 2024 23:00:32 GMT"} })
699	2024-12-20T23:00:32.898Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
700	2024-12-20T23:00:32.898Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
701	2024-12-20T23:00:32.902Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
702	2024-12-20T23:00:32.902Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
703	2024-12-20T23:00:32.907Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
704	2024-12-20T23:00:32.907Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
705	2024-12-20T23:00:32.908Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
706	2024-12-20T23:00:32.908Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
707	2024-12-20T23:00:32.915Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 530.409964ms iteration = 14 status = {}
708	2024-12-20T23:00:33.149Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 295 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 94b953c3-3b4d-44ca-a425-ca6070786097 response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
709	2024-12-20T23:00:42.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 41 reason = Timeout
710	2024-12-20T23:00:42.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 41 reason = Timeout
711	2024-12-20T23:00:42.385Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
712	2024-12-20T23:00:42.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
713	2024-12-20T23:00:42.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
714	2024-12-20T23:00:42.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
715	2024-12-20T23:00:42.387Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
716	2024-12-20T23:00:42.388Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
717	2024-12-20T23:00:42.388Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
718	2024-12-20T23:00:42.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
719	2024-12-20T23:00:42.389Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
720	2024-12-20T23:00:42.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
721	2024-12-20T23:00:42.391Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
722	2024-12-20T23:00:42.391Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
723	2024-12-20T23:00:42.393Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
724	2024-12-20T23:00:42.395Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
725	2024-12-20T23:00:42.395Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
726	2024-12-20T23:00:42.395Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
727	2024-12-20T23:00:42.406Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
728	2024-12-20T23:00:42.406Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.201603ms iteration = 41 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
729	2024-12-20T23:00:42.416Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.290499ms iteration = 41 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
730	2024-12-20T23:00:43.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 294 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = dc8f860c-d4b4-4fe6-88c5-ec5658974adb response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
731	2024-12-20T23:00:52.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 42 reason = Timeout
732	2024-12-20T23:00:52.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
733	2024-12-20T23:00:52.384Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 42 reason = Timeout
734	2024-12-20T23:00:52.384Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
735	2024-12-20T23:00:52.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
736	2024-12-20T23:00:52.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
737	2024-12-20T23:00:52.387Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
738	2024-12-20T23:00:52.387Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
739	2024-12-20T23:00:52.387Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
740	2024-12-20T23:00:52.389Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.7118ms iteration = 42 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
741	2024-12-20T23:00:52.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
742	2024-12-20T23:00:52.390Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
743	2024-12-20T23:00:52.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
744	2024-12-20T23:00:52.392Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
745	2024-12-20T23:00:52.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
746	2024-12-20T23:00:52.394Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
747	2024-12-20T23:00:52.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
748	2024-12-20T23:00:52.396Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
749	2024-12-20T23:00:52.397Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
750	2024-12-20T23:00:52.408Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
751	2024-12-20T23:00:52.408Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.934447ms iteration = 42 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
752	2024-12-20T23:00:53.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 255 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 10d6f8e3-75e9-4036-9462-918484a4711b response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
753	2024-12-20T23:00:58.138Z	INFO	nexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:34496
754	2024-12-20T23:00:58.138Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34496 req_id = db3bee4f-d076-40c2-a046-396753dc52ee roles = RoleSet { roles: {} } uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?sort_by=id_ascending
755	2024-12-20T23:00:58.138Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34496 req_id = db3bee4f-d076-40c2-a046-396753dc52ee resource = Database result = Ok(()) uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?sort_by=id_ascending
756	2024-12-20T23:00:58.140Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2009 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34496 req_id = db3bee4f-d076-40c2-a046-396753dc52ee response_code = 200 uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?sort_by=id_ascending
757	2024-12-20T23:00:58.140Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34496 req_id = fc652285-7e6f-428a-ab81-71b1c78c2c34 roles = RoleSet { roles: {} } uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmM2I2MGJkMy02ZTM0LTQxNDAtYTc1NS01N2YyNmRiYzViMzQifX0%3D
758	2024-12-20T23:00:58.140Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34496 req_id = fc652285-7e6f-428a-ab81-71b1c78c2c34 resource = Database result = Ok(()) uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmM2I2MGJkMy02ZTM0LTQxNDAtYTc1NS01N2YyNmRiYzViMzQifX0%3D
759	2024-12-20T23:00:58.142Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2144 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34496 req_id = fc652285-7e6f-428a-ab81-71b1c78c2c34 response_code = 200 uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmM2I2MGJkMy02ZTM0LTQxNDAtYTc1NS01N2YyNmRiYzViMzQifX0%3D
760	2024-12-20T23:01:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 15 reason = Timeout
761	2024-12-20T23:01:02.384Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
762	2024-12-20T23:01:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 15 reason = Timeout
763	2024-12-20T23:01:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
764	2024-12-20T23:01:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
765	2024-12-20T23:01:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 8 reason = Timeout
766	2024-12-20T23:01:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 15 reason = Timeout
767	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
768	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
769	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 8 reason = Timeout
770	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
771	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 15 reason = Timeout
772	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
773	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 15 reason = Timeout
774	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 8 reason = Timeout
775	2024-12-20T23:01:02.385Z	INFO	nexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
776	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
777	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
778	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 15 reason = Timeout
779	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 15 reason = Timeout
780	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
781	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
782	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 10 reason = Timeout
783	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 43 reason = Timeout
784	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
785	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 15 reason = Timeout
786	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
787	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 15 reason = Timeout
788	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
789	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 9 reason = Timeout
790	2024-12-20T23:01:02.385Z	WARN	nexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7
791	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 12.801\u{b5}s iteration = 9 status = {"enabled":false,"target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7"}
792	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 8 reason = Timeout
793	2024-12-20T23:01:02.385Z	INFO	nexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-20 22:51:02.385112103 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
794	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 8 reason = Timeout
795	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
796	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
797	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 8 reason = Timeout
798	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
799	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
800	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
801	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = external_endpoints iteration = 8 reason = Timeout
802	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 15 reason = Timeout
803	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
804	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
805	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 8 reason = Timeout
806	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 15 reason = Timeout
807	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
808	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 15 reason = Timeout
809	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
810	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 8 reason = Timeout
811	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 15 reason = Timeout
812	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 16 reason = Timeout
813	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
814	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 8 reason = Timeout
815	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
816	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
817	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 10 reason = Timeout
818	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 15 reason = Timeout
819	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 43 reason = Timeout
820	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
821	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
822	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
823	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
824	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
825	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
826	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
827	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
828	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
829	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
830	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
831	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
832	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
833	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
834	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
835	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
836	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
837	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
838	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
839	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
840	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
841	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
842	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
843	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
844	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
845	2024-12-20T23:01:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
846	2024-12-20T23:01:02.385Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1e296b8d-6800-4056-8034-a1ec2d005fff.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("ace0592c-7755-46f8-b47a-c9b6444fe64c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("ace0592c-7755-46f8-b47a-c9b6444fe64c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1077, tv_nsec: 775585231 } })
847	2024-12-20T23:01:02.385Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("616ce87c-1f39-49d9-ba35-ee63b2650ecd.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b4615881-2f68-4c82-be17-071bf3ae3641.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c9217e92-0af3-4930-acb2-aa82ded5b782.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("c9217e92-0af3-4930-acb2-aa82ded5b782.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 1077, tv_nsec: 775666383 } })
848	2024-12-20T23:01:02.386Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
849	2024-12-20T23:01:02.386Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.09098ms iteration = 8 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
850	2024-12-20T23:01:02.386Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
851	2024-12-20T23:01:02.386Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.325087ms iteration = 8 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
852	2024-12-20T23:01:02.388Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
853	2024-12-20T23:01:02.388Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 3.683001ms iteration = 15 status = {"errors":[],"finish_invoked_ok":[]}
854	2024-12-20T23:01:02.389Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
855	2024-12-20T23:01:02.390Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.968387ms iteration = 15 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
856	2024-12-20T23:01:02.390Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
857	2024-12-20T23:01:02.390Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
858	2024-12-20T23:01:02.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
859	2024-12-20T23:01:02.391Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
860	2024-12-20T23:01:02.391Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
861	2024-12-20T23:01:02.391Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
862	2024-12-20T23:01:02.392Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 7.548487ms iteration = 15 status = {"errors":[],"garbage_collect_requested":[]}
863	2024-12-20T23:01:02.393Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
864	2024-12-20T23:01:02.394Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
865	2024-12-20T23:01:02.394Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
866	2024-12-20T23:01:02.394Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
867	2024-12-20T23:01:02.396Z	INFO	nexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-20 22:51:02.385112103 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
868	2024-12-20T23:01:02.396Z	DEBG	nexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 10.991272ms iteration = 8 status = {"expiration":"2024-12-20T22:51:02.385112103Z","pruned":[]}
869	2024-12-20T23:01:02.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
870	2024-12-20T23:01:02.396Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
871	2024-12-20T23:01:02.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
872	2024-12-20T23:01:02.397Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
873	2024-12-20T23:01:02.397Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
874	2024-12-20T23:01:02.397Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
875	2024-12-20T23:01:02.398Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
876	2024-12-20T23:01:02.398Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
877	2024-12-20T23:01:02.399Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
878	2024-12-20T23:01:02.403Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1077, tv_nsec: 793454362 } })
879	2024-12-20T23:01:02.404Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
880	2024-12-20T23:01:02.404Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
881	2024-12-20T23:01:02.404Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
882	2024-12-20T23:01:02.404Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
883	2024-12-20T23:01:02.404Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
884	2024-12-20T23:01:02.404Z	DEBG	nexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 19.66582ms iteration = 8 status = {"errors":[],"found_port_ok":[]}
885	2024-12-20T23:01:02.406Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
886	2024-12-20T23:01:02.406Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
887	2024-12-20T23:01:02.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
888	2024-12-20T23:01:02.407Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
889	2024-12-20T23:01:02.407Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
890	2024-12-20T23:01:02.407Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
891	2024-12-20T23:01:02.407Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
892	2024-12-20T23:01:02.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
893	2024-12-20T23:01:02.408Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1077, tv_nsec: 798720656 } })
894	2024-12-20T23:01:02.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
895	2024-12-20T23:01:02.409Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
896	2024-12-20T23:01:02.409Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
897	2024-12-20T23:01:02.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
898	2024-12-20T23:01:02.412Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
899	2024-12-20T23:01:02.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
900	2024-12-20T23:01:02.412Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1077, tv_nsec: 802611333 } })
901	2024-12-20T23:01:02.412Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
902	2024-12-20T23:01:02.413Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
903	2024-12-20T23:01:02.415Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
904	2024-12-20T23:01:02.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
905	2024-12-20T23:01:02.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
906	2024-12-20T23:01:02.417Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
907	2024-12-20T23:01:02.417Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
908	2024-12-20T23:01:02.417Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
909	2024-12-20T23:01:02.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
910	2024-12-20T23:01:02.418Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
911	2024-12-20T23:01:02.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
912	2024-12-20T23:01:02.418Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
913	2024-12-20T23:01:02.418Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
914	2024-12-20T23:01:02.418Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
915	2024-12-20T23:01:02.418Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
916	2024-12-20T23:01:02.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
917	2024-12-20T23:01:02.419Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
918	2024-12-20T23:01:02.420Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
919	2024-12-20T23:01:02.423Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
920	2024-12-20T23:01:02.423Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
921	2024-12-20T23:01:02.423Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
922	2024-12-20T23:01:02.423Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
923	2024-12-20T23:01:02.423Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
924	2024-12-20T23:01:02.423Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
925	2024-12-20T23:01:02.435Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
926	2024-12-20T23:01:02.435Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
927	2024-12-20T23:01:02.437Z	DEBG	nexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
928	2024-12-20T23:01:02.437Z	INFO	nexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
929	2024-12-20T23:01:02.437Z	DEBG	nexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 52.346377ms iteration = 8 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
930	2024-12-20T23:01:02.437Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 52.261015ms iteration = 43 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
931	2024-12-20T23:01:02.444Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
932	2024-12-20T23:01:02.444Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
933	2024-12-20T23:01:02.444Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
934	2024-12-20T23:01:02.444Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
935	2024-12-20T23:01:02.450Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
936	2024-12-20T23:01:02.450Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
937	2024-12-20T23:01:02.451Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
938	2024-12-20T23:01:02.451Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
939	2024-12-20T23:01:02.455Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
940	2024-12-20T23:01:02.455Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
941	2024-12-20T23:01:02.458Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
942	2024-12-20T23:01:02.458Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
943	2024-12-20T23:01:02.458Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
944	2024-12-20T23:01:02.458Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
945	2024-12-20T23:01:02.458Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
946	2024-12-20T23:01:02.459Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
947	2024-12-20T23:01:02.460Z	DEBG	nexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
948	2024-12-20T23:01:02.460Z	INFO	nexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
949	2024-12-20T23:01:02.460Z	DEBG	nexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 75.723019ms iteration = 8 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
950	2024-12-20T23:01:02.461Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
951	2024-12-20T23:01:02.463Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
952	2024-12-20T23:01:02.463Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
953	2024-12-20T23:01:02.463Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
954	2024-12-20T23:01:02.466Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
955	2024-12-20T23:01:02.466Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
956	2024-12-20T23:01:02.467Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
957	2024-12-20T23:01:02.467Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 82.29963ms iteration = 15 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
958	2024-12-20T23:01:02.468Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
959	2024-12-20T23:01:02.470Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
960	2024-12-20T23:01:02.470Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
961	2024-12-20T23:01:02.471Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
962	2024-12-20T23:01:02.471Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
963	2024-12-20T23:01:02.471Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
964	2024-12-20T23:01:02.471Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 86.616818ms iteration = 15 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
965	2024-12-20T23:01:02.475Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
966	2024-12-20T23:01:02.475Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
967	2024-12-20T23:01:02.475Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
968	2024-12-20T23:01:02.476Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
969	2024-12-20T23:01:02.476Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
970	2024-12-20T23:01:02.478Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
971	2024-12-20T23:01:02.478Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
972	2024-12-20T23:01:02.479Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
973	2024-12-20T23:01:02.484Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
974	2024-12-20T23:01:02.486Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
975	2024-12-20T23:01:02.486Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
976	2024-12-20T23:01:02.486Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
977	2024-12-20T23:01:02.489Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
978	2024-12-20T23:01:02.490Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 105.62586ms iteration = 15 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
979	2024-12-20T23:01:02.491Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
980	2024-12-20T23:01:02.492Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 287d6774-d3e6-4b73-8c77-c90534d4a54d, lookup_type: ById(287d6774-d3e6-4b73-8c77-c90534d4a54d) } result = Ok(())
981	2024-12-20T23:01:02.492Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
982	2024-12-20T23:01:02.492Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
983	2024-12-20T23:01:02.492Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
984	2024-12-20T23:01:02.494Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
985	2024-12-20T23:01:02.494Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 108.89477ms iteration = 43 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
986	2024-12-20T23:01:02.494Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 109.439525ms iteration = 15 status = 0
987	2024-12-20T23:01:02.503Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 118.474143ms iteration = 15 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
988	2024-12-20T23:01:02.510Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
989	2024-12-20T23:01:02.510Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
990	2024-12-20T23:01:02.510Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
991	2024-12-20T23:01:02.510Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
992	2024-12-20T23:01:02.510Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
993	2024-12-20T23:01:02.512Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
994	2024-12-20T23:01:02.515Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
995	2024-12-20T23:01:02.515Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
996	2024-12-20T23:01:02.515Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
997	2024-12-20T23:01:02.517Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
998	2024-12-20T23:01:02.517Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 1e86bf23-6ffd-4a12-93c7-0061d7d17590 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
999	2024-12-20T23:01:02.517Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1000	2024-12-20T23:01:02.517Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1001	2024-12-20T23:01:02.519Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1002	2024-12-20T23:01:02.522Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1003	2024-12-20T23:01:02.522Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1004	2024-12-20T23:01:02.522Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1005	2024-12-20T23:01:02.525Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 1e86bf23-6ffd-4a12-93c7-0061d7d17590 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
1006	2024-12-20T23:01:02.525Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
1007	2024-12-20T23:01:02.525Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-20 22:54:38.956071 UTC dns_group = internal generation = 1
1008	2024-12-20T23:01:02.525Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 140.333123ms iteration = 8 status = {"generation":1}
1009	2024-12-20T23:01:02.525Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
1010	2024-12-20T23:01:02.525Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 5fa20c62-75c0-4531-be07-51b87d303d18 dns_zone_name = oxide.test found_so_far = 0 version = 2
1011	2024-12-20T23:01:02.525Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1012	2024-12-20T23:01:02.526Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1013	2024-12-20T23:01:02.528Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1014	2024-12-20T23:01:02.529Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
1015	2024-12-20T23:01:02.530Z	DEBG	nexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-20T22:53:58.073373Z, time_modified: 2024-12-20T22:53:58.073373Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, db_silo: Silo { identity: SiloIdentity { id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-20T22:54:39.847015Z, time_modified: 2024-12-20T22:54:39.847015Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, db_silo: Silo { identity: SiloIdentity { id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-20T22:54:39.847015Z, time_modified: 2024-12-20T22:54:39.847015Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156" }] }) }
1016	2024-12-20T23:01:02.530Z	DEBG	nexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 145.68577ms iteration = 8 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"ee579470-b16b-4af1-ad3a-b3d29f9a2fb2","tls_certs":["ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156"]}},"default_endpoint":{"silo_id":"ee579470-b16b-4af1-ad3a-b3d29f9a2fb2","tls_certs":["ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
1017	2024-12-20T23:01:02.532Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1018	2024-12-20T23:01:02.532Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1019	2024-12-20T23:01:02.532Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1020	2024-12-20T23:01:02.534Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1021	2024-12-20T23:01:02.534Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 5fa20c62-75c0-4531-be07-51b87d303d18 dns_zone_name = oxide.test found_so_far = 1 version = 2
1022	2024-12-20T23:01:02.534Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
1023	2024-12-20T23:01:02.534Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-20 22:54:39.899407 UTC dns_group = external generation = 2
1024	2024-12-20T23:01:02.534Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 149.542626ms iteration = 8 status = {"generation":2}
1025	2024-12-20T23:01:02.534Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1026	2024-12-20T23:01:02.540Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 155.823509ms iteration = 15 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
1027	2024-12-20T23:01:02.544Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 159.241943ms iteration = 16 status = {"physical_disks_added":0}
1028	2024-12-20T23:01:02.569Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
1029	2024-12-20T23:01:02.569Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1030	2024-12-20T23:01:02.569Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "833f468d-f3a7-4b59-b29e-a9366f7f8f89", "content-length": "26", "date": "Fri, 20 Dec 2024 23:01:02 GMT"} })
1031	2024-12-20T23:01:02.569Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1032	2024-12-20T23:01:02.570Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1033	2024-12-20T23:01:02.570Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1034	2024-12-20T23:01:02.571Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1035	2024-12-20T23:01:02.571Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
1036	2024-12-20T23:01:02.573Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
1037	2024-12-20T23:01:02.573Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1038	2024-12-20T23:01:02.573Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1039	2024-12-20T23:01:02.575Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1040	2024-12-20T23:01:02.576Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
1041	2024-12-20T23:01:02.576Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1042	2024-12-20T23:01:02.576Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1043	2024-12-20T23:01:02.577Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 zone_address = fd00:1122:3344:101::2
1044	2024-12-20T23:01:02.577Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1045	2024-12-20T23:01:02.578Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f2ef6cab-4e72-4b87-a736-0eda2e78993f", "content-length": "26", "date": "Fri, 20 Dec 2024 23:01:02 GMT"} })
1046	2024-12-20T23:01:02.578Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1047	2024-12-20T23:01:02.579Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
1048	2024-12-20T23:01:02.581Z	WARN	nexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
1049	2024-12-20T23:01:02.581Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 196.728451ms iteration = 8 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
1050	2024-12-20T23:01:02.600Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
1051	2024-12-20T23:01:02.714Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
1052	2024-12-20T23:01:02.714Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
1053	2024-12-20T23:01:02.731Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
1054	2024-12-20T23:01:02.731Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1055	2024-12-20T23:01:02.731Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5683719c-c733-4c30-b167-91073dd77e8d", "content-length": "26", "date": "Fri, 20 Dec 2024 23:01:02 GMT"} })
1056	2024-12-20T23:01:02.731Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1057	2024-12-20T23:01:02.733Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
1058	2024-12-20T23:01:02.741Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a878e0f-f862-4d8f-ac10-350a78dfd4dc", "content-length": "372", "date": "Fri, 20 Dec 2024 23:01:02 GMT"} })
1059	2024-12-20T23:01:02.741Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1060	2024-12-20T23:01:02.744Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1061	2024-12-20T23:01:02.760Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
1062	2024-12-20T23:01:02.785Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
1063	2024-12-20T23:01:02.785Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "78bdfc62-b520-4fa6-9abf-78b5a0dc40c8", "content-length": "691", "date": "Fri, 20 Dec 2024 23:01:02 GMT"} })
1064	2024-12-20T23:01:02.785Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
1065	2024-12-20T23:01:02.785Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
1066	2024-12-20T23:01:02.785Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
1067	2024-12-20T23:01:02.787Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 402.509762ms iteration = 15 status = {}
1068	2024-12-20T23:01:02.824Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
1069	2024-12-20T23:01:02.824Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
1070	2024-12-20T23:01:02.824Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "90c69f35-daf2-4293-b47c-1c4e90178e1a", "date": "Fri, 20 Dec 2024 23:01:02 GMT"} })
1071	2024-12-20T23:01:02.824Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1072	2024-12-20T23:01:02.824Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1073	2024-12-20T23:01:02.827Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1074	2024-12-20T23:01:02.828Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1075	2024-12-20T23:01:02.831Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1076	2024-12-20T23:01:02.831Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1077	2024-12-20T23:01:02.835Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
1078	2024-12-20T23:01:02.835Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
1079	2024-12-20T23:01:02.835Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "296ac51c-e177-4dd8-836d-8d8cf47a9233", "date": "Fri, 20 Dec 2024 23:01:02 GMT"} })
1080	2024-12-20T23:01:02.838Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 453.520032ms iteration = 15 status = {}
1081	2024-12-20T23:01:02.842Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
1082	2024-12-20T23:01:02.842Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "0e4e62d4-fe06-46ec-9973-73086824e159", "date": "Fri, 20 Dec 2024 23:01:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
1083	2024-12-20T23:01:02.842Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
1084	2024-12-20T23:01:02.844Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "df88217d-bb49-4f81-a901-115e11a9a2aa", "date": "Fri, 20 Dec 2024 23:01:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
1085	2024-12-20T23:01:02.844Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
1086	2024-12-20T23:01:02.846Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
1087	2024-12-20T23:01:02.846Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 461.480952ms iteration = 10 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
1088	2024-12-20T23:01:02.885Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
1089	2024-12-20T23:01:02.885Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb595628-563b-4a61-9c7a-566488dc57f4", "content-length": "1", "date": "Fri, 20 Dec 2024 23:01:02 GMT"} })
1090	2024-12-20T23:01:02.887Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
1091	2024-12-20T23:01:02.887Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1092	2024-12-20T23:01:02.888Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 503.527706ms iteration = 15 status = null
1093	2024-12-20T23:01:02.904Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
1094	2024-12-20T23:01:02.904Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "4d70c557-ddb7-4658-8050-cfb3a7922e28", "date": "Fri, 20 Dec 2024 23:01:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
1095	2024-12-20T23:01:02.904Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
1096	2024-12-20T23:01:02.906Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 521.678825ms iteration = 15 status = {}
1097	2024-12-20T23:01:02.907Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "c0016fab-26d3-4825-b3e2-8b9191372b67", "date": "Fri, 20 Dec 2024 23:01:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
1098	2024-12-20T23:01:02.907Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
1099	2024-12-20T23:01:02.909Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "2db7e7e3-37cc-47c1-9cf8-b6fdc34076cf", "date": "Fri, 20 Dec 2024 23:01:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
1100	2024-12-20T23:01:02.909Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
1101	2024-12-20T23:01:02.911Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
1102	2024-12-20T23:01:02.911Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 525.915551ms iteration = 10 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
1103	2024-12-20T23:01:02.964Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1104	2024-12-20T23:01:02.964Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1105	2024-12-20T23:01:02.966Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 03dd12a0-dc2d-46e1-a7f4-d3397b9b7aa7, rack_id: b394d0c6-19a8-4136-9148-af6c47c1f8e0, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(560ca89c-7c0c-4921-8e5c-b57676f695b0) } rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
1106	2024-12-20T23:01:02.966Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1107	2024-12-20T23:01:02.966Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1108	2024-12-20T23:01:02.976Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
1109	2024-12-20T23:01:02.977Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "0afe3651-4654-4cb2-835e-67370c1483d5", "content-length": "134", "date": "Fri, 20 Dec 2024 23:01:02 GMT"} })
1110	2024-12-20T23:01:02.977Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
1111	2024-12-20T23:01:02.977Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 560ca89c-7c0c-4921-8e5c-b57676f695b0
1112	2024-12-20T23:01:02.977Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
1113	2024-12-20T23:01:02.977Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
1114	2024-12-20T23:01:02.977Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
1115	2024-12-20T23:01:02.977Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
1116	2024-12-20T23:01:02.977Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
1117	2024-12-20T23:01:02.977Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
1118	2024-12-20T23:01:02.977Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
1119	2024-12-20T23:01:02.977Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
1120	2024-12-20T23:01:02.977Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
1121	2024-12-20T23:01:02.977Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1122	2024-12-20T23:01:02.978Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1123	2024-12-20T23:01:02.980Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
1124	2024-12-20T23:01:02.980Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ddfb72b-af5c-4c95-93c3-b503c425bf06", "content-length": "2", "date": "Fri, 20 Dec 2024 23:01:02 GMT"} })
1125	2024-12-20T23:01:02.980Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
1126	2024-12-20T23:01:02.980Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "e48011e8-8adf-486c-beb3-b70e5a01fda7", "content-length": "52", "date": "Fri, 20 Dec 2024 23:01:02 GMT"} })
1127	2024-12-20T23:01:02.980Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
1128	2024-12-20T23:01:02.980Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
1129	2024-12-20T23:01:03.078Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "1ba83e8b-95d3-4633-9080-61de9518f02c", "date": "Fri, 20 Dec 2024 23:01:02 GMT"} })
1130	2024-12-20T23:01:03.078Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch = Switch0
1131	2024-12-20T23:01:03.078Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
1132	2024-12-20T23:01:03.078Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "7404c5d8-1b17-4116-a6e6-858da6aa27da", "content-length": "682", "date": "Fri, 20 Dec 2024 23:01:02 GMT"} })
1133	2024-12-20T23:01:03.078Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1134	2024-12-20T23:01:03.078Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1135	2024-12-20T23:01:03.082Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1136	2024-12-20T23:01:03.082Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1137	2024-12-20T23:01:03.086Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1138	2024-12-20T23:01:03.086Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1139	2024-12-20T23:01:03.088Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1140	2024-12-20T23:01:03.088Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1141	2024-12-20T23:01:03.096Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 711.394984ms iteration = 15 status = {}
1142	2024-12-20T23:01:03.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 230 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 789a3803-cc24-4afe-a2ec-59c19376112f response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
1143	2024-12-20T23:01:12.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 44 reason = Timeout
1144	2024-12-20T23:01:12.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 44 reason = Timeout
1145	2024-12-20T23:01:12.385Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1146	2024-12-20T23:01:12.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1147	2024-12-20T23:01:12.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1148	2024-12-20T23:01:12.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1149	2024-12-20T23:01:12.387Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1150	2024-12-20T23:01:12.388Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1151	2024-12-20T23:01:12.388Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1152	2024-12-20T23:01:12.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1153	2024-12-20T23:01:12.390Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1154	2024-12-20T23:01:12.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1155	2024-12-20T23:01:12.390Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 5.038309ms iteration = 44 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
1156	2024-12-20T23:01:12.391Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1157	2024-12-20T23:01:12.391Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1158	2024-12-20T23:01:12.393Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1159	2024-12-20T23:01:12.395Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
1160	2024-12-20T23:01:12.395Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1161	2024-12-20T23:01:12.395Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1162	2024-12-20T23:01:12.404Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
1163	2024-12-20T23:01:12.404Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.403952ms iteration = 44 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
1164	2024-12-20T23:01:13.139Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34496 req_id = 9861c8fb-d2b5-4f18-8ef6-2fea1287d4ea roles = RoleSet { roles: {} } uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?sort_by=id_ascending
1165	2024-12-20T23:01:13.139Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34496 req_id = 9861c8fb-d2b5-4f18-8ef6-2fea1287d4ea resource = Database result = Ok(()) uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?sort_by=id_ascending
1166	2024-12-20T23:01:13.141Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2022 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34496 req_id = 9861c8fb-d2b5-4f18-8ef6-2fea1287d4ea response_code = 200 uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?sort_by=id_ascending
1167	2024-12-20T23:01:13.141Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34496 req_id = e12ced2b-f5c3-4b47-badb-dcc28bdd3617 roles = RoleSet { roles: {} } uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmM2I2MGJkMy02ZTM0LTQxNDAtYTc1NS01N2YyNmRiYzViMzQifX0%3D
1168	2024-12-20T23:01:13.141Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34496 req_id = e12ced2b-f5c3-4b47-badb-dcc28bdd3617 resource = Database result = Ok(()) uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmM2I2MGJkMy02ZTM0LTQxNDAtYTc1NS01N2YyNmRiYzViMzQifX0%3D
1169	2024-12-20T23:01:13.143Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2161 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34496 req_id = e12ced2b-f5c3-4b47-badb-dcc28bdd3617 response_code = 200 uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmM2I2MGJkMy02ZTM0LTQxNDAtYTc1NS01N2YyNmRiYzViMzQifX0%3D
1170	2024-12-20T23:01:13.149Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 304 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = d0730f4a-1b28-486b-9501-19dd9b63914e response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
1171	2024-12-20T23:01:22.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 45 reason = Timeout
1172	2024-12-20T23:01:22.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 45 reason = Timeout
1173	2024-12-20T23:01:22.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1174	2024-12-20T23:01:22.385Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1175	2024-12-20T23:01:22.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1176	2024-12-20T23:01:22.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1177	2024-12-20T23:01:22.387Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1178	2024-12-20T23:01:22.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1179	2024-12-20T23:01:22.390Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1180	2024-12-20T23:01:22.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1181	2024-12-20T23:01:22.392Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1182	2024-12-20T23:01:22.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1183	2024-12-20T23:01:22.397Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1184	2024-12-20T23:01:22.400Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
1185	2024-12-20T23:01:22.400Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1186	2024-12-20T23:01:22.400Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1187	2024-12-20T23:01:22.413Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
1188	2024-12-20T23:01:22.413Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.456072ms iteration = 45 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
1189	2024-12-20T23:01:22.420Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1190	2024-12-20T23:01:22.420Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1191	2024-12-20T23:01:22.449Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 63.803682ms iteration = 45 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
1192	2024-12-20T23:01:23.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 283 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = aa73313d-f8a0-4313-8a3c-bf6673298cd2 response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
1193	2024-12-20T23:01:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 16 reason = Timeout
1194	2024-12-20T23:01:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
1195	2024-12-20T23:01:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 16 reason = Timeout
1196	2024-12-20T23:01:32.384Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
1197	2024-12-20T23:01:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 16 reason = Timeout
1198	2024-12-20T23:01:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
1199	2024-12-20T23:01:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
1200	2024-12-20T23:01:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 16 reason = Timeout
1201	2024-12-20T23:01:32.384Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
1202	2024-12-20T23:01:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 17 reason = Timeout
1203	2024-12-20T23:01:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 16 reason = Timeout
1204	2024-12-20T23:01:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1205	2024-12-20T23:01:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1206	2024-12-20T23:01:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1207	2024-12-20T23:01:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 46 reason = Timeout
1208	2024-12-20T23:01:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1209	2024-12-20T23:01:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 16 reason = Timeout
1210	2024-12-20T23:01:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 16 reason = Timeout
1211	2024-12-20T23:01:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1212	2024-12-20T23:01:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
1213	2024-12-20T23:01:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 16 reason = Timeout
1214	2024-12-20T23:01:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
1215	2024-12-20T23:01:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 16 reason = Timeout
1216	2024-12-20T23:01:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
1217	2024-12-20T23:01:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 46 reason = Timeout
1218	2024-12-20T23:01:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1219	2024-12-20T23:01:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 16 reason = Timeout
1220	2024-12-20T23:01:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1221	2024-12-20T23:01:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 16 reason = Timeout
1222	2024-12-20T23:01:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
1223	2024-12-20T23:01:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 16 reason = Timeout
1224	2024-12-20T23:01:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 16 reason = Timeout
1225	2024-12-20T23:01:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
1226	2024-12-20T23:01:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 16 reason = Timeout
1227	2024-12-20T23:01:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
1228	2024-12-20T23:01:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
1229	2024-12-20T23:01:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1230	2024-12-20T23:01:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1231	2024-12-20T23:01:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1232	2024-12-20T23:01:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1233	2024-12-20T23:01:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1234	2024-12-20T23:01:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
1235	2024-12-20T23:01:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
1236	2024-12-20T23:01:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
1237	2024-12-20T23:01:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1238	2024-12-20T23:01:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
1239	2024-12-20T23:01:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1240	2024-12-20T23:01:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1241	2024-12-20T23:01:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
1242	2024-12-20T23:01:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1243	2024-12-20T23:01:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
1244	2024-12-20T23:01:32.387Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.604132ms iteration = 16 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
1245	2024-12-20T23:01:32.388Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1107, tv_nsec: 778114540 } })
1246	2024-12-20T23:01:32.388Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1247	2024-12-20T23:01:32.388Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1248	2024-12-20T23:01:32.388Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
1249	2024-12-20T23:01:32.389Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1250	2024-12-20T23:01:32.389Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 4.931085ms iteration = 16 status = {"errors":[],"garbage_collect_requested":[]}
1251	2024-12-20T23:01:32.390Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1252	2024-12-20T23:01:32.390Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
1253	2024-12-20T23:01:32.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
1254	2024-12-20T23:01:32.391Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
1255	2024-12-20T23:01:32.391Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1256	2024-12-20T23:01:32.391Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1257	2024-12-20T23:01:32.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1258	2024-12-20T23:01:32.392Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1259	2024-12-20T23:01:32.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1260	2024-12-20T23:01:32.392Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
1261	2024-12-20T23:01:32.392Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1262	2024-12-20T23:01:32.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1263	2024-12-20T23:01:32.394Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
1264	2024-12-20T23:01:32.394Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1265	2024-12-20T23:01:32.394Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
1266	2024-12-20T23:01:32.394Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
1267	2024-12-20T23:01:32.394Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
1268	2024-12-20T23:01:32.394Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1269	2024-12-20T23:01:32.394Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
1270	2024-12-20T23:01:32.394Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1271	2024-12-20T23:01:32.394Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1107, tv_nsec: 784473755 } })
1272	2024-12-20T23:01:32.395Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
1273	2024-12-20T23:01:32.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
1274	2024-12-20T23:01:32.396Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1275	2024-12-20T23:01:32.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1276	2024-12-20T23:01:32.396Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1277	2024-12-20T23:01:32.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1278	2024-12-20T23:01:32.398Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 13.147261ms iteration = 16 status = {"errors":[],"finish_invoked_ok":[]}
1279	2024-12-20T23:01:32.398Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1280	2024-12-20T23:01:32.398Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
1281	2024-12-20T23:01:32.398Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 13.595433ms iteration = 16 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
1282	2024-12-20T23:01:32.398Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
1283	2024-12-20T23:01:32.399Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1284	2024-12-20T23:01:32.399Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1285	2024-12-20T23:01:32.399Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1286	2024-12-20T23:01:32.399Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1287	2024-12-20T23:01:32.399Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1288	2024-12-20T23:01:32.399Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
1289	2024-12-20T23:01:32.399Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
1290	2024-12-20T23:01:32.400Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
1291	2024-12-20T23:01:32.401Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1292	2024-12-20T23:01:32.401Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
1293	2024-12-20T23:01:32.401Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1294	2024-12-20T23:01:32.401Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1295	2024-12-20T23:01:32.401Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1296	2024-12-20T23:01:32.404Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
1297	2024-12-20T23:01:32.404Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
1298	2024-12-20T23:01:32.404Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
1299	2024-12-20T23:01:32.404Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
1300	2024-12-20T23:01:32.406Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 21.071059ms iteration = 46 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
1301	2024-12-20T23:01:32.406Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 21.659355ms iteration = 16 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
1302	2024-12-20T23:01:32.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1303	2024-12-20T23:01:32.407Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1107, tv_nsec: 797212525 } })
1304	2024-12-20T23:01:32.407Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
1305	2024-12-20T23:01:32.408Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1306	2024-12-20T23:01:32.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
1307	2024-12-20T23:01:32.410Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1308	2024-12-20T23:01:32.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1309	2024-12-20T23:01:32.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
1310	2024-12-20T23:01:32.410Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1311	2024-12-20T23:01:32.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1312	2024-12-20T23:01:32.434Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1313	2024-12-20T23:01:32.434Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1314	2024-12-20T23:01:32.440Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1315	2024-12-20T23:01:32.440Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
1316	2024-12-20T23:01:32.440Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1317	2024-12-20T23:01:32.440Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1318	2024-12-20T23:01:32.445Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
1319	2024-12-20T23:01:32.445Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
1320	2024-12-20T23:01:32.445Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 60.746288ms iteration = 16 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
1321	2024-12-20T23:01:32.447Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
1322	2024-12-20T23:01:32.447Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 62.556067ms iteration = 16 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
1323	2024-12-20T23:01:32.549Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 zone_address = fd00:1122:3344:101::2
1324	2024-12-20T23:01:32.549Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1325	2024-12-20T23:01:32.549Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "02b90a21-a0f6-49aa-ba6c-f233baaa6eb8", "content-length": "26", "date": "Fri, 20 Dec 2024 23:01:32 GMT"} })
1326	2024-12-20T23:01:32.549Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1327	2024-12-20T23:01:32.549Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
1328	2024-12-20T23:01:32.549Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
1329	2024-12-20T23:01:32.550Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "a18068e1-8378-4dd7-8f77-8729a46e8e3b", "content-length": "372", "date": "Fri, 20 Dec 2024 23:01:32 GMT"} })
1330	2024-12-20T23:01:32.550Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1331	2024-12-20T23:01:32.550Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1332	2024-12-20T23:01:32.550Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
1333	2024-12-20T23:01:32.552Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1334	2024-12-20T23:01:32.553Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1335	2024-12-20T23:01:32.558Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
1336	2024-12-20T23:01:32.558Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 174.05427ms iteration = 46 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
1337	2024-12-20T23:01:32.559Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1338	2024-12-20T23:01:32.559Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1339	2024-12-20T23:01:32.560Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1340	2024-12-20T23:01:32.562Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
1341	2024-12-20T23:01:32.562Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1342	2024-12-20T23:01:32.562Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1343	2024-12-20T23:01:32.564Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1344	2024-12-20T23:01:32.565Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
1345	2024-12-20T23:01:32.565Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1346	2024-12-20T23:01:32.565Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1347	2024-12-20T23:01:32.576Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
1348	2024-12-20T23:01:32.576Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1349	2024-12-20T23:01:32.577Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d482bcf5-2195-4f3a-b627-5545c822677c", "content-length": "26", "date": "Fri, 20 Dec 2024 23:01:32 GMT"} })
1350	2024-12-20T23:01:32.577Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1351	2024-12-20T23:01:32.578Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
1352	2024-12-20T23:01:32.584Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
1353	2024-12-20T23:01:32.584Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1354	2024-12-20T23:01:32.587Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 202.978243ms iteration = 17 status = {"physical_disks_added":0}
1355	2024-12-20T23:01:32.588Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1356	2024-12-20T23:01:32.591Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 287d6774-d3e6-4b73-8c77-c90534d4a54d, lookup_type: ById(287d6774-d3e6-4b73-8c77-c90534d4a54d) } result = Ok(())
1357	2024-12-20T23:01:32.591Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1358	2024-12-20T23:01:32.591Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1359	2024-12-20T23:01:32.593Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 208.525125ms iteration = 16 status = 0
1360	2024-12-20T23:01:32.602Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
1361	2024-12-20T23:01:32.602Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
1362	2024-12-20T23:01:32.603Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "199b3a9b-82ff-42af-b310-a9ce9348982c", "date": "Fri, 20 Dec 2024 23:01:32 GMT"} })
1363	2024-12-20T23:01:32.603Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1364	2024-12-20T23:01:32.603Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1365	2024-12-20T23:01:32.604Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1366	2024-12-20T23:01:32.605Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1367	2024-12-20T23:01:32.613Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 228.220907ms iteration = 16 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
1368	2024-12-20T23:01:32.649Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
1369	2024-12-20T23:01:32.649Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1370	2024-12-20T23:01:32.649Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d73d5e3d-34e7-494c-a50a-e9c029ead176", "content-length": "26", "date": "Fri, 20 Dec 2024 23:01:32 GMT"} })
1371	2024-12-20T23:01:32.649Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1372	2024-12-20T23:01:32.651Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
1373	2024-12-20T23:01:32.690Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
1374	2024-12-20T23:01:32.691Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2a148f7-bfea-4aa6-b031-87b296f15915", "content-length": "691", "date": "Fri, 20 Dec 2024 23:01:32 GMT"} })
1375	2024-12-20T23:01:32.691Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
1376	2024-12-20T23:01:32.691Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
1377	2024-12-20T23:01:32.691Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
1378	2024-12-20T23:01:32.692Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 307.606327ms iteration = 16 status = {}
1379	2024-12-20T23:01:32.693Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1380	2024-12-20T23:01:32.694Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1381	2024-12-20T23:01:32.697Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
1382	2024-12-20T23:01:32.697Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
1383	2024-12-20T23:01:32.697Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "10c60ab3-e486-4385-8f5d-8b6eaa3a8f11", "date": "Fri, 20 Dec 2024 23:01:32 GMT"} })
1384	2024-12-20T23:01:32.698Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 314.146996ms iteration = 16 status = {}
1385	2024-12-20T23:01:32.718Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
1386	2024-12-20T23:01:32.718Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "daad0aba-909d-4773-b274-316bf4a026d8", "content-length": "1", "date": "Fri, 20 Dec 2024 23:01:32 GMT"} })
1387	2024-12-20T23:01:32.719Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
1388	2024-12-20T23:01:32.719Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1389	2024-12-20T23:01:32.722Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 337.431965ms iteration = 16 status = null
1390	2024-12-20T23:01:32.821Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 436.057074ms iteration = 16 status = {}
1391	2024-12-20T23:01:32.846Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1392	2024-12-20T23:01:32.846Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1393	2024-12-20T23:01:32.847Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 03dd12a0-dc2d-46e1-a7f4-d3397b9b7aa7, rack_id: b394d0c6-19a8-4136-9148-af6c47c1f8e0, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(560ca89c-7c0c-4921-8e5c-b57676f695b0) } rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
1394	2024-12-20T23:01:32.847Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1395	2024-12-20T23:01:32.848Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1396	2024-12-20T23:01:32.856Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
1397	2024-12-20T23:01:32.856Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "5cee05f3-b2db-4323-89a4-118efdb4507d", "content-length": "134", "date": "Fri, 20 Dec 2024 23:01:32 GMT"} })
1398	2024-12-20T23:01:32.856Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
1399	2024-12-20T23:01:32.856Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 560ca89c-7c0c-4921-8e5c-b57676f695b0
1400	2024-12-20T23:01:32.857Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
1401	2024-12-20T23:01:32.857Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
1402	2024-12-20T23:01:32.857Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
1403	2024-12-20T23:01:32.857Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
1404	2024-12-20T23:01:32.857Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
1405	2024-12-20T23:01:32.857Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
1406	2024-12-20T23:01:32.857Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
1407	2024-12-20T23:01:32.857Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
1408	2024-12-20T23:01:32.857Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
1409	2024-12-20T23:01:32.857Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1410	2024-12-20T23:01:32.857Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1411	2024-12-20T23:01:32.860Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
1412	2024-12-20T23:01:32.860Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "4aed17fb-ab32-46aa-ba12-4cb29af20311", "content-length": "2", "date": "Fri, 20 Dec 2024 23:01:32 GMT"} })
1413	2024-12-20T23:01:32.860Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
1414	2024-12-20T23:01:32.860Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "8db5906a-799f-43a8-bd10-8297edd3ee3b", "content-length": "52", "date": "Fri, 20 Dec 2024 23:01:32 GMT"} })
1415	2024-12-20T23:01:32.860Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
1416	2024-12-20T23:01:32.860Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
1417	2024-12-20T23:01:32.972Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "65a70271-aa7d-45e9-91ca-5fc705212354", "date": "Fri, 20 Dec 2024 23:01:32 GMT"} })
1418	2024-12-20T23:01:32.972Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch = Switch0
1419	2024-12-20T23:01:32.972Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
1420	2024-12-20T23:01:32.972Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "d19a57fa-f0a4-4ed9-9d9e-8b5b902c14d5", "content-length": "682", "date": "Fri, 20 Dec 2024 23:01:32 GMT"} })
1421	2024-12-20T23:01:32.972Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1422	2024-12-20T23:01:32.973Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1423	2024-12-20T23:01:32.976Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1424	2024-12-20T23:01:32.976Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1425	2024-12-20T23:01:32.978Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1426	2024-12-20T23:01:32.979Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1427	2024-12-20T23:01:32.980Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1428	2024-12-20T23:01:32.980Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1429	2024-12-20T23:01:32.987Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 602.758351ms iteration = 16 status = {}
1430	2024-12-20T23:01:33.149Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 335 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 644fe3ab-3f11-405a-92e2-aec83ee90fdb response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
1431	2024-12-20T23:01:42.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 47 reason = Timeout
1432	2024-12-20T23:01:42.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 47 reason = Timeout
1433	2024-12-20T23:01:42.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1434	2024-12-20T23:01:42.385Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1435	2024-12-20T23:01:42.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1436	2024-12-20T23:01:42.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1437	2024-12-20T23:01:42.387Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1438	2024-12-20T23:01:42.387Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1439	2024-12-20T23:01:42.388Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1440	2024-12-20T23:01:42.388Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.64648ms iteration = 47 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
1441	2024-12-20T23:01:42.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1442	2024-12-20T23:01:42.390Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1443	2024-12-20T23:01:42.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1444	2024-12-20T23:01:42.392Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1445	2024-12-20T23:01:42.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1446	2024-12-20T23:01:42.394Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1447	2024-12-20T23:01:42.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
1448	2024-12-20T23:01:42.396Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1449	2024-12-20T23:01:42.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1450	2024-12-20T23:01:42.405Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
1451	2024-12-20T23:01:42.405Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.848056ms iteration = 47 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
1452	2024-12-20T23:01:43.138Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34496 req_id = 8794d57b-3044-41e1-9fb9-1fff2298085c roles = RoleSet { roles: {} } uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?sort_by=id_ascending
1453	2024-12-20T23:01:43.138Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34496 req_id = 8794d57b-3044-41e1-9fb9-1fff2298085c resource = Database result = Ok(()) uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?sort_by=id_ascending
1454	2024-12-20T23:01:43.141Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3194 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34496 req_id = 8794d57b-3044-41e1-9fb9-1fff2298085c response_code = 200 uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?sort_by=id_ascending
1455	2024-12-20T23:01:43.141Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34496 req_id = be17f385-a5b2-4d9c-9f50-073ba922968a roles = RoleSet { roles: {} } uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmM2I2MGJkMy02ZTM0LTQxNDAtYTc1NS01N2YyNmRiYzViMzQifX0%3D
1456	2024-12-20T23:01:43.142Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34496 req_id = be17f385-a5b2-4d9c-9f50-073ba922968a resource = Database result = Ok(()) uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmM2I2MGJkMy02ZTM0LTQxNDAtYTc1NS01N2YyNmRiYzViMzQifX0%3D
1457	2024-12-20T23:01:43.143Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1988 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34496 req_id = be17f385-a5b2-4d9c-9f50-073ba922968a response_code = 200 uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmM2I2MGJkMy02ZTM0LTQxNDAtYTc1NS01N2YyNmRiYzViMzQifX0%3D
1458	2024-12-20T23:01:43.149Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 293 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 91647ab0-e103-4f83-84e4-bc197f881df4 response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
1459	2024-12-20T23:01:52.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 48 reason = Timeout
1460	2024-12-20T23:01:52.385Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1461	2024-12-20T23:01:52.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 48 reason = Timeout
1462	2024-12-20T23:01:52.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1463	2024-12-20T23:01:52.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1464	2024-12-20T23:01:52.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1465	2024-12-20T23:01:52.387Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1466	2024-12-20T23:01:52.387Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1467	2024-12-20T23:01:52.388Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1468	2024-12-20T23:01:52.389Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.161985ms iteration = 48 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
1469	2024-12-20T23:01:52.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1470	2024-12-20T23:01:52.390Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1471	2024-12-20T23:01:52.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1472	2024-12-20T23:01:52.392Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1473	2024-12-20T23:01:52.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1474	2024-12-20T23:01:52.394Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1475	2024-12-20T23:01:52.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
1476	2024-12-20T23:01:52.396Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1477	2024-12-20T23:01:52.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1478	2024-12-20T23:01:52.405Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
1479	2024-12-20T23:01:52.405Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.459842ms iteration = 48 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
1480	2024-12-20T23:01:53.149Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 414 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 75cb7004-fbbd-487f-98d6-db79026d15da response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
1481	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 17 reason = Timeout
1482	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
1483	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 17 reason = Timeout
1484	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
1485	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
1486	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 17 reason = Timeout
1487	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 9 reason = Timeout
1488	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
1489	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
1490	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 9 reason = Timeout
1491	2024-12-20T23:02:02.385Z	INFO	nexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-20 22:52:02.385165093 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
1492	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 9 reason = Timeout
1493	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
1494	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
1495	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 17 reason = Timeout
1496	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 17 reason = Timeout
1497	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 17 reason = Timeout
1498	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1499	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 17 reason = Timeout
1500	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
1501	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 17 reason = Timeout
1502	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
1503	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
1504	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 9 reason = Timeout
1505	2024-12-20T23:02:02.385Z	INFO	nexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
1506	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 10 reason = Timeout
1507	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
1508	2024-12-20T23:02:02.385Z	WARN	nexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7
1509	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 17 reason = Timeout
1510	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 15.231\u{b5}s iteration = 10 status = {"enabled":false,"target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7"}
1511	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 9 reason = Timeout
1512	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
1513	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
1514	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 49 reason = Timeout
1515	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1516	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1517	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 18 reason = Timeout
1518	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1519	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 49 reason = Timeout
1520	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1521	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 11 reason = Timeout
1522	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 17 reason = Timeout
1523	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1524	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 17 reason = Timeout
1525	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1526	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 9 reason = Timeout
1527	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 9 reason = Timeout
1528	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
1529	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 9 reason = Timeout
1530	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 17 reason = Timeout
1531	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
1532	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 17 reason = Timeout
1533	2024-12-20T23:02:02.385Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
1534	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1535	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = external_endpoints iteration = 9 reason = Timeout
1536	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 9 reason = Timeout
1537	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1538	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
1539	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1540	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 17 reason = Timeout
1541	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 11 reason = Timeout
1542	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
1543	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
1544	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
1545	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1546	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
1547	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
1548	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1549	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
1550	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
1551	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
1552	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1553	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
1554	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1555	2024-12-20T23:02:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1556	2024-12-20T23:02:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1557	2024-12-20T23:02:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1558	2024-12-20T23:02:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1559	2024-12-20T23:02:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
1560	2024-12-20T23:02:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
1561	2024-12-20T23:02:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1562	2024-12-20T23:02:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1563	2024-12-20T23:02:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1564	2024-12-20T23:02:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1565	2024-12-20T23:02:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
1566	2024-12-20T23:02:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
1567	2024-12-20T23:02:02.386Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("616ce87c-1f39-49d9-ba35-ee63b2650ecd.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b4615881-2f68-4c82-be17-071bf3ae3641.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c9217e92-0af3-4930-acb2-aa82ded5b782.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("c9217e92-0af3-4930-acb2-aa82ded5b782.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 1137, tv_nsec: 775803537 } })
1568	2024-12-20T23:02:02.386Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1e296b8d-6800-4056-8034-a1ec2d005fff.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("ace0592c-7755-46f8-b47a-c9b6444fe64c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("ace0592c-7755-46f8-b47a-c9b6444fe64c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1137, tv_nsec: 775811447 } })
1569	2024-12-20T23:02:02.386Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
1570	2024-12-20T23:02:02.386Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.222084ms iteration = 9 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
1571	2024-12-20T23:02:02.386Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
1572	2024-12-20T23:02:02.386Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.220923ms iteration = 9 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
1573	2024-12-20T23:02:02.388Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.470905ms iteration = 17 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
1574	2024-12-20T23:02:02.389Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1137, tv_nsec: 779090737 } })
1575	2024-12-20T23:02:02.389Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
1576	2024-12-20T23:02:02.390Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
1577	2024-12-20T23:02:02.390Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
1578	2024-12-20T23:02:02.390Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
1579	2024-12-20T23:02:02.390Z	DEBG	nexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
1580	2024-12-20T23:02:02.390Z	INFO	nexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
1581	2024-12-20T23:02:02.390Z	DEBG	nexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 5.09144ms iteration = 9 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
1582	2024-12-20T23:02:02.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
1583	2024-12-20T23:02:02.390Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1584	2024-12-20T23:02:02.391Z	INFO	nexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-20 22:52:02.385165093 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
1585	2024-12-20T23:02:02.391Z	DEBG	nexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 6.103668ms iteration = 9 status = {"expiration":"2024-12-20T22:52:02.385165093Z","pruned":[]}
1586	2024-12-20T23:02:02.391Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1587	2024-12-20T23:02:02.391Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1588	2024-12-20T23:02:02.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
1589	2024-12-20T23:02:02.392Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1590	2024-12-20T23:02:02.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1591	2024-12-20T23:02:02.394Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1592	2024-12-20T23:02:02.395Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1593	2024-12-20T23:02:02.395Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1594	2024-12-20T23:02:02.395Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
1595	2024-12-20T23:02:02.395Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1596	2024-12-20T23:02:02.395Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1597	2024-12-20T23:02:02.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
1598	2024-12-20T23:02:02.396Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
1599	2024-12-20T23:02:02.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
1600	2024-12-20T23:02:02.397Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
1601	2024-12-20T23:02:02.397Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1602	2024-12-20T23:02:02.397Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1603	2024-12-20T23:02:02.398Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1604	2024-12-20T23:02:02.399Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1605	2024-12-20T23:02:02.399Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1606	2024-12-20T23:02:02.400Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1137, tv_nsec: 789791831 } })
1607	2024-12-20T23:02:02.400Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
1608	2024-12-20T23:02:02.401Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
1609	2024-12-20T23:02:02.401Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1610	2024-12-20T23:02:02.402Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1611	2024-12-20T23:02:02.402Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1612	2024-12-20T23:02:02.402Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1613	2024-12-20T23:02:02.402Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1614	2024-12-20T23:02:02.402Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
1615	2024-12-20T23:02:02.402Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1616	2024-12-20T23:02:02.402Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1617	2024-12-20T23:02:02.403Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1618	2024-12-20T23:02:02.403Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1619	2024-12-20T23:02:02.403Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1620	2024-12-20T23:02:02.404Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1621	2024-12-20T23:02:02.404Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1622	2024-12-20T23:02:02.405Z	DEBG	nexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 20.284777ms iteration = 9 status = {"errors":[],"found_port_ok":[]}
1623	2024-12-20T23:02:02.406Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1624	2024-12-20T23:02:02.406Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
1625	2024-12-20T23:02:02.406Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1626	2024-12-20T23:02:02.406Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
1627	2024-12-20T23:02:02.406Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
1628	2024-12-20T23:02:02.406Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
1629	2024-12-20T23:02:02.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1630	2024-12-20T23:02:02.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
1631	2024-12-20T23:02:02.407Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1137, tv_nsec: 797263046 } })
1632	2024-12-20T23:02:02.407Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
1633	2024-12-20T23:02:02.408Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
1634	2024-12-20T23:02:02.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1635	2024-12-20T23:02:02.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1636	2024-12-20T23:02:02.410Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
1637	2024-12-20T23:02:02.410Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
1638	2024-12-20T23:02:02.410Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
1639	2024-12-20T23:02:02.410Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1640	2024-12-20T23:02:02.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1641	2024-12-20T23:02:02.411Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1642	2024-12-20T23:02:02.412Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1643	2024-12-20T23:02:02.412Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1644	2024-12-20T23:02:02.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1645	2024-12-20T23:02:02.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1646	2024-12-20T23:02:02.413Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1647	2024-12-20T23:02:02.413Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1648	2024-12-20T23:02:02.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1649	2024-12-20T23:02:02.414Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
1650	2024-12-20T23:02:02.414Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1651	2024-12-20T23:02:02.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1652	2024-12-20T23:02:02.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1653	2024-12-20T23:02:02.417Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 32.339638ms iteration = 17 status = {"errors":[],"finish_invoked_ok":[]}
1654	2024-12-20T23:02:02.418Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1655	2024-12-20T23:02:02.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1656	2024-12-20T23:02:02.418Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
1657	2024-12-20T23:02:02.418Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
1658	2024-12-20T23:02:02.418Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 33.553371ms iteration = 17 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
1659	2024-12-20T23:02:02.418Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1660	2024-12-20T23:02:02.419Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1661	2024-12-20T23:02:02.420Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1662	2024-12-20T23:02:02.422Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
1663	2024-12-20T23:02:02.422Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1664	2024-12-20T23:02:02.422Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1665	2024-12-20T23:02:02.423Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1666	2024-12-20T23:02:02.423Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1667	2024-12-20T23:02:02.423Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1668	2024-12-20T23:02:02.424Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
1669	2024-12-20T23:02:02.424Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1670	2024-12-20T23:02:02.424Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1671	2024-12-20T23:02:02.425Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1672	2024-12-20T23:02:02.429Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
1673	2024-12-20T23:02:02.429Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1674	2024-12-20T23:02:02.429Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1675	2024-12-20T23:02:02.438Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
1676	2024-12-20T23:02:02.438Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
1677	2024-12-20T23:02:02.439Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
1678	2024-12-20T23:02:02.439Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
1679	2024-12-20T23:02:02.439Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
1680	2024-12-20T23:02:02.439Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 54.518937ms iteration = 49 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
1681	2024-12-20T23:02:02.443Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 58.148267ms iteration = 49 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
1682	2024-12-20T23:02:02.444Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1683	2024-12-20T23:02:02.446Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
1684	2024-12-20T23:02:02.446Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
1685	2024-12-20T23:02:02.447Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
1686	2024-12-20T23:02:02.447Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 61.710564ms iteration = 17 status = {"errors":[],"garbage_collect_requested":[]}
1687	2024-12-20T23:02:02.450Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1688	2024-12-20T23:02:02.450Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1689	2024-12-20T23:02:02.466Z	DEBG	nexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
1690	2024-12-20T23:02:02.466Z	INFO	nexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
1691	2024-12-20T23:02:02.466Z	DEBG	nexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 81.474007ms iteration = 9 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
1692	2024-12-20T23:02:02.470Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 84.984474ms iteration = 17 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
1693	2024-12-20T23:02:02.471Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
1694	2024-12-20T23:02:02.472Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
1695	2024-12-20T23:02:02.473Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
1696	2024-12-20T23:02:02.473Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
1697	2024-12-20T23:02:02.473Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
1698	2024-12-20T23:02:02.473Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1699	2024-12-20T23:02:02.473Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1700	2024-12-20T23:02:02.478Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
1701	2024-12-20T23:02:02.478Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 1e86bf23-6ffd-4a12-93c7-0061d7d17590 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
1702	2024-12-20T23:02:02.478Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1703	2024-12-20T23:02:02.479Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1704	2024-12-20T23:02:02.481Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 96.354606ms iteration = 18 status = {"physical_disks_added":0}
1705	2024-12-20T23:02:02.518Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1706	2024-12-20T23:02:02.518Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1707	2024-12-20T23:02:02.521Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
1708	2024-12-20T23:02:02.553Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
1709	2024-12-20T23:02:02.553Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1710	2024-12-20T23:02:02.554Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c38171d-8c8f-4603-8e55-421661aa033f", "content-length": "26", "date": "Fri, 20 Dec 2024 23:02:02 GMT"} })
1711	2024-12-20T23:02:02.554Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1712	2024-12-20T23:02:02.556Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 zone_address = fd00:1122:3344:101::2
1713	2024-12-20T23:02:02.556Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1714	2024-12-20T23:02:02.556Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
1715	2024-12-20T23:02:02.556Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e047f27-8612-4891-a102-8058e26f3f12", "content-length": "26", "date": "Fri, 20 Dec 2024 23:02:02 GMT"} })
1716	2024-12-20T23:02:02.556Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1717	2024-12-20T23:02:02.559Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
1718	2024-12-20T23:02:02.583Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
1719	2024-12-20T23:02:02.583Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
1720	2024-12-20T23:02:02.585Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 200.224428ms iteration = 17 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
1721	2024-12-20T23:02:02.587Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1722	2024-12-20T23:02:02.588Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1723	2024-12-20T23:02:02.588Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1724	2024-12-20T23:02:02.588Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1725	2024-12-20T23:02:02.588Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1726	2024-12-20T23:02:02.591Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
1727	2024-12-20T23:02:02.591Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1728	2024-12-20T23:02:02.591Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1729	2024-12-20T23:02:02.591Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1730	2024-12-20T23:02:02.591Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1731	2024-12-20T23:02:02.591Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1732	2024-12-20T23:02:02.592Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1733	2024-12-20T23:02:02.592Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1734	2024-12-20T23:02:02.592Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1735	2024-12-20T23:02:02.593Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
1736	2024-12-20T23:02:02.593Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 207.97915ms iteration = 17 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
1737	2024-12-20T23:02:02.597Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 1e86bf23-6ffd-4a12-93c7-0061d7d17590 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
1738	2024-12-20T23:02:02.597Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
1739	2024-12-20T23:02:02.597Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-20 22:54:38.956071 UTC dns_group = internal generation = 1
1740	2024-12-20T23:02:02.597Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 211.366244ms iteration = 9 status = {"generation":1}
1741	2024-12-20T23:02:02.618Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1742	2024-12-20T23:02:02.619Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1743	2024-12-20T23:02:02.620Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1744	2024-12-20T23:02:02.622Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
1745	2024-12-20T23:02:02.622Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1746	2024-12-20T23:02:02.622Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1747	2024-12-20T23:02:02.631Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1748	2024-12-20T23:02:02.632Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 246.996712ms iteration = 17 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
1749	2024-12-20T23:02:02.633Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
1750	2024-12-20T23:02:02.633Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1751	2024-12-20T23:02:02.633Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1752	2024-12-20T23:02:02.633Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1753	2024-12-20T23:02:02.635Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
1754	2024-12-20T23:02:02.635Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 5fa20c62-75c0-4531-be07-51b87d303d18 dns_zone_name = oxide.test found_so_far = 0 version = 2
1755	2024-12-20T23:02:02.635Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1756	2024-12-20T23:02:02.636Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1757	2024-12-20T23:02:02.636Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 287d6774-d3e6-4b73-8c77-c90534d4a54d, lookup_type: ById(287d6774-d3e6-4b73-8c77-c90534d4a54d) } result = Ok(())
1758	2024-12-20T23:02:02.636Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1759	2024-12-20T23:02:02.636Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1760	2024-12-20T23:02:02.638Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1761	2024-12-20T23:02:02.638Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 253.502551ms iteration = 17 status = 0
1762	2024-12-20T23:02:02.641Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1763	2024-12-20T23:02:02.641Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1764	2024-12-20T23:02:02.641Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1765	2024-12-20T23:02:02.644Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 5fa20c62-75c0-4531-be07-51b87d303d18 dns_zone_name = oxide.test found_so_far = 1 version = 2
1766	2024-12-20T23:02:02.644Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
1767	2024-12-20T23:02:02.644Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-20 22:54:39.899407 UTC dns_group = external generation = 2
1768	2024-12-20T23:02:02.644Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 258.519969ms iteration = 9 status = {"generation":2}
1769	2024-12-20T23:02:02.654Z	WARN	nexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
1770	2024-12-20T23:02:02.654Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 268.906844ms iteration = 9 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
1771	2024-12-20T23:02:02.659Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "4afe4950-7b01-4c09-bd78-810cc92ab79d", "content-length": "372", "date": "Fri, 20 Dec 2024 23:02:02 GMT"} })
1772	2024-12-20T23:02:02.659Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1773	2024-12-20T23:02:02.659Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1774	2024-12-20T23:02:02.663Z	DEBG	nexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-20T22:53:58.073373Z, time_modified: 2024-12-20T22:53:58.073373Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, db_silo: Silo { identity: SiloIdentity { id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-20T22:54:39.847015Z, time_modified: 2024-12-20T22:54:39.847015Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, db_silo: Silo { identity: SiloIdentity { id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-20T22:54:39.847015Z, time_modified: 2024-12-20T22:54:39.847015Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156" }] }) }
1775	2024-12-20T23:02:02.663Z	DEBG	nexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 278.22621ms iteration = 9 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"ee579470-b16b-4af1-ad3a-b3d29f9a2fb2","tls_certs":["ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156"]}},"default_endpoint":{"silo_id":"ee579470-b16b-4af1-ad3a-b3d29f9a2fb2","tls_certs":["ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
1776	2024-12-20T23:02:02.689Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
1777	2024-12-20T23:02:02.706Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
1778	2024-12-20T23:02:02.706Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
1779	2024-12-20T23:02:02.707Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "1ca4740e-891d-45c1-bf8f-642af2a508ee", "date": "Fri, 20 Dec 2024 23:02:02 GMT"} })
1780	2024-12-20T23:02:02.707Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1781	2024-12-20T23:02:02.707Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
1782	2024-12-20T23:02:02.707Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1783	2024-12-20T23:02:02.707Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1784	2024-12-20T23:02:02.707Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "fffc0d23-4a87-4c20-ae51-48152c3f0221", "content-length": "26", "date": "Fri, 20 Dec 2024 23:02:02 GMT"} })
1785	2024-12-20T23:02:02.707Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1786	2024-12-20T23:02:02.709Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
1787	2024-12-20T23:02:02.731Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
1788	2024-12-20T23:02:02.873Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
1789	2024-12-20T23:02:02.873Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "f0bfe3d3-18f7-4c5e-885f-2249c1e00977", "date": "Fri, 20 Dec 2024 23:02:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
1790	2024-12-20T23:02:02.873Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
1791	2024-12-20T23:02:02.874Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "a17819e8-d260-4f8f-b034-9e33398c8c9d", "date": "Fri, 20 Dec 2024 23:02:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
1792	2024-12-20T23:02:02.874Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
1793	2024-12-20T23:02:02.876Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "f75eb0d1-4833-4f5d-a6f2-c472f212480d", "date": "Fri, 20 Dec 2024 23:02:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
1794	2024-12-20T23:02:02.876Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
1795	2024-12-20T23:02:02.877Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
1796	2024-12-20T23:02:02.878Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a2a75e7-5c34-4977-8e0c-c34c09d3d677", "content-length": "691", "date": "Fri, 20 Dec 2024 23:02:02 GMT"} })
1797	2024-12-20T23:02:02.878Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
1798	2024-12-20T23:02:02.878Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
1799	2024-12-20T23:02:02.878Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
1800	2024-12-20T23:02:02.878Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
1801	2024-12-20T23:02:02.878Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 492.970036ms iteration = 11 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
1802	2024-12-20T23:02:02.879Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1803	2024-12-20T23:02:02.879Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 494.504538ms iteration = 17 status = {}
1804	2024-12-20T23:02:02.880Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1805	2024-12-20T23:02:02.883Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1806	2024-12-20T23:02:02.883Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1807	2024-12-20T23:02:02.884Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
1808	2024-12-20T23:02:02.884Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab5b5436-d4cc-4018-9c41-6dfcfdd060b4", "content-length": "1", "date": "Fri, 20 Dec 2024 23:02:02 GMT"} })
1809	2024-12-20T23:02:02.886Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
1810	2024-12-20T23:02:02.887Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
1811	2024-12-20T23:02:02.887Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
1812	2024-12-20T23:02:02.887Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1813	2024-12-20T23:02:02.887Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "031cf404-80e9-4db9-913a-140efb8baf71", "date": "Fri, 20 Dec 2024 23:02:02 GMT"} })
1814	2024-12-20T23:02:02.889Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 503.597068ms iteration = 17 status = {}
1815	2024-12-20T23:02:02.889Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 504.156653ms iteration = 17 status = null
1816	2024-12-20T23:02:02.897Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 511.641409ms iteration = 17 status = {}
1817	2024-12-20T23:02:02.920Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
1818	2024-12-20T23:02:02.920Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "34f57041-b56b-4f14-a2b9-e656c1cc09d0", "date": "Fri, 20 Dec 2024 23:02:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
1819	2024-12-20T23:02:02.920Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
1820	2024-12-20T23:02:02.921Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "3fb32ff5-bc3f-4c86-95a7-fcffb16ae306", "date": "Fri, 20 Dec 2024 23:02:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
1821	2024-12-20T23:02:02.921Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
1822	2024-12-20T23:02:02.924Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
1823	2024-12-20T23:02:02.924Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 538.529067ms iteration = 11 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
1824	2024-12-20T23:02:02.934Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1825	2024-12-20T23:02:02.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1826	2024-12-20T23:02:02.937Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 03dd12a0-dc2d-46e1-a7f4-d3397b9b7aa7, rack_id: b394d0c6-19a8-4136-9148-af6c47c1f8e0, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(560ca89c-7c0c-4921-8e5c-b57676f695b0) } rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
1827	2024-12-20T23:02:02.937Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1828	2024-12-20T23:02:02.937Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1829	2024-12-20T23:02:02.947Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
1830	2024-12-20T23:02:02.947Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "bbdf1eaf-318e-4408-9bef-ecb68d2a2861", "content-length": "134", "date": "Fri, 20 Dec 2024 23:02:02 GMT"} })
1831	2024-12-20T23:02:02.947Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
1832	2024-12-20T23:02:02.947Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 560ca89c-7c0c-4921-8e5c-b57676f695b0
1833	2024-12-20T23:02:02.948Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
1834	2024-12-20T23:02:02.948Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
1835	2024-12-20T23:02:02.948Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
1836	2024-12-20T23:02:02.948Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
1837	2024-12-20T23:02:02.948Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
1838	2024-12-20T23:02:02.948Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
1839	2024-12-20T23:02:02.948Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
1840	2024-12-20T23:02:02.948Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
1841	2024-12-20T23:02:02.948Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
1842	2024-12-20T23:02:02.948Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1843	2024-12-20T23:02:02.948Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1844	2024-12-20T23:02:02.950Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
1845	2024-12-20T23:02:02.950Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "36d2a289-35ac-4060-a7c3-668cde8527e9", "content-length": "2", "date": "Fri, 20 Dec 2024 23:02:02 GMT"} })
1846	2024-12-20T23:02:02.950Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
1847	2024-12-20T23:02:02.951Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "45756e66-4e23-449e-aea5-af9e5f2a7815", "content-length": "52", "date": "Fri, 20 Dec 2024 23:02:02 GMT"} })
1848	2024-12-20T23:02:02.951Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
1849	2024-12-20T23:02:02.951Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
1850	2024-12-20T23:02:03.049Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "a4ab746a-a08e-4927-bdfa-bec9d5148bca", "date": "Fri, 20 Dec 2024 23:02:02 GMT"} })
1851	2024-12-20T23:02:03.049Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch = Switch0
1852	2024-12-20T23:02:03.049Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
1853	2024-12-20T23:02:03.049Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef590ffc-8122-4a27-90c8-ac859d03354f", "content-length": "682", "date": "Fri, 20 Dec 2024 23:02:02 GMT"} })
1854	2024-12-20T23:02:03.049Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1855	2024-12-20T23:02:03.049Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1856	2024-12-20T23:02:03.053Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1857	2024-12-20T23:02:03.053Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1858	2024-12-20T23:02:03.057Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1859	2024-12-20T23:02:03.057Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1860	2024-12-20T23:02:03.059Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1861	2024-12-20T23:02:03.060Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1862	2024-12-20T23:02:03.068Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 683.015095ms iteration = 17 status = {}
1863	2024-12-20T23:02:03.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 268 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 6bf54c96-c93c-4d2c-840c-d8f38cf2ae3e response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
1864	2024-12-20T23:02:12.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 50 reason = Timeout
1865	2024-12-20T23:02:12.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1866	2024-12-20T23:02:12.384Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 50 reason = Timeout
1867	2024-12-20T23:02:12.384Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1868	2024-12-20T23:02:12.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1869	2024-12-20T23:02:12.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1870	2024-12-20T23:02:12.387Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1871	2024-12-20T23:02:12.387Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1872	2024-12-20T23:02:12.387Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1873	2024-12-20T23:02:12.388Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.242327ms iteration = 50 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
1874	2024-12-20T23:02:12.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1875	2024-12-20T23:02:12.389Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1876	2024-12-20T23:02:12.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1877	2024-12-20T23:02:12.391Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1878	2024-12-20T23:02:12.391Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1879	2024-12-20T23:02:12.393Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1880	2024-12-20T23:02:12.395Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
1881	2024-12-20T23:02:12.395Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1882	2024-12-20T23:02:12.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1883	2024-12-20T23:02:12.408Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
1884	2024-12-20T23:02:12.408Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.499645ms iteration = 50 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
1885	2024-12-20T23:02:13.149Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 266 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = b572c253-4d13-4344-a1d1-38c6272368c5 response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
1886	2024-12-20T23:02:22.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 51 reason = Timeout
1887	2024-12-20T23:02:22.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 51 reason = Timeout
1888	2024-12-20T23:02:22.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1889	2024-12-20T23:02:22.385Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1890	2024-12-20T23:02:22.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1891	2024-12-20T23:02:22.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1892	2024-12-20T23:02:22.387Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1893	2024-12-20T23:02:22.388Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1894	2024-12-20T23:02:22.388Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1895	2024-12-20T23:02:22.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1896	2024-12-20T23:02:22.390Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1897	2024-12-20T23:02:22.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1898	2024-12-20T23:02:22.392Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1899	2024-12-20T23:02:22.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1900	2024-12-20T23:02:22.394Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1901	2024-12-20T23:02:22.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
1902	2024-12-20T23:02:22.396Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1903	2024-12-20T23:02:22.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1904	2024-12-20T23:02:22.407Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
1905	2024-12-20T23:02:22.407Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.719306ms iteration = 51 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
1906	2024-12-20T23:02:22.423Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 37.636523ms iteration = 51 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
1907	2024-12-20T23:02:23.149Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 423 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 7fec8f07-59e7-4ec8-8722-7dbc5c658d9f response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
1908	2024-12-20T23:02:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 18 reason = Timeout
1909	2024-12-20T23:02:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
1910	2024-12-20T23:02:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 18 reason = Timeout
1911	2024-12-20T23:02:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 18 reason = Timeout
1912	2024-12-20T23:02:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1913	2024-12-20T23:02:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
1914	2024-12-20T23:02:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 18 reason = Timeout
1915	2024-12-20T23:02:32.385Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
1916	2024-12-20T23:02:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 18 reason = Timeout
1917	2024-12-20T23:02:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
1918	2024-12-20T23:02:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
1919	2024-12-20T23:02:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 18 reason = Timeout
1920	2024-12-20T23:02:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 19 reason = Timeout
1921	2024-12-20T23:02:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 52 reason = Timeout
1922	2024-12-20T23:02:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1923	2024-12-20T23:02:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 18 reason = Timeout
1924	2024-12-20T23:02:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
1925	2024-12-20T23:02:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 18 reason = Timeout
1926	2024-12-20T23:02:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1927	2024-12-20T23:02:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
1928	2024-12-20T23:02:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 52 reason = Timeout
1929	2024-12-20T23:02:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 18 reason = Timeout
1930	2024-12-20T23:02:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
1931	2024-12-20T23:02:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1932	2024-12-20T23:02:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 18 reason = Timeout
1933	2024-12-20T23:02:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1934	2024-12-20T23:02:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 18 reason = Timeout
1935	2024-12-20T23:02:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
1936	2024-12-20T23:02:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 18 reason = Timeout
1937	2024-12-20T23:02:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 18 reason = Timeout
1938	2024-12-20T23:02:32.385Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
1939	2024-12-20T23:02:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1940	2024-12-20T23:02:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1941	2024-12-20T23:02:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 18 reason = Timeout
1942	2024-12-20T23:02:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
1943	2024-12-20T23:02:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
1944	2024-12-20T23:02:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1945	2024-12-20T23:02:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
1946	2024-12-20T23:02:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1947	2024-12-20T23:02:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
1948	2024-12-20T23:02:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
1949	2024-12-20T23:02:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1950	2024-12-20T23:02:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
1951	2024-12-20T23:02:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1952	2024-12-20T23:02:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1953	2024-12-20T23:02:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1954	2024-12-20T23:02:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
1955	2024-12-20T23:02:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1956	2024-12-20T23:02:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
1957	2024-12-20T23:02:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1958	2024-12-20T23:02:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1959	2024-12-20T23:02:32.388Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.995682ms iteration = 18 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
1960	2024-12-20T23:02:32.389Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1961	2024-12-20T23:02:32.389Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1167, tv_nsec: 779406386 } })
1962	2024-12-20T23:02:32.390Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
1963	2024-12-20T23:02:32.390Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1964	2024-12-20T23:02:32.391Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1965	2024-12-20T23:02:32.391Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
1966	2024-12-20T23:02:32.391Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
1967	2024-12-20T23:02:32.391Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
1968	2024-12-20T23:02:32.391Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1969	2024-12-20T23:02:32.391Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1970	2024-12-20T23:02:32.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
1971	2024-12-20T23:02:32.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1972	2024-12-20T23:02:32.393Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
1973	2024-12-20T23:02:32.393Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1974	2024-12-20T23:02:32.393Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
1975	2024-12-20T23:02:32.393Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
1976	2024-12-20T23:02:32.393Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1977	2024-12-20T23:02:32.393Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
1978	2024-12-20T23:02:32.393Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1979	2024-12-20T23:02:32.394Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1980	2024-12-20T23:02:32.395Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
1981	2024-12-20T23:02:32.395Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1982	2024-12-20T23:02:32.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1983	2024-12-20T23:02:32.396Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1167, tv_nsec: 785754390 } })
1984	2024-12-20T23:02:32.396Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
1985	2024-12-20T23:02:32.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
1986	2024-12-20T23:02:32.396Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1987	2024-12-20T23:02:32.397Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1988	2024-12-20T23:02:32.400Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1989	2024-12-20T23:02:32.400Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1990	2024-12-20T23:02:32.400Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1991	2024-12-20T23:02:32.401Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1992	2024-12-20T23:02:32.401Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1993	2024-12-20T23:02:32.401Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1994	2024-12-20T23:02:32.401Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1995	2024-12-20T23:02:32.402Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
1996	2024-12-20T23:02:32.402Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1997	2024-12-20T23:02:32.402Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1998	2024-12-20T23:02:32.403Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1999	2024-12-20T23:02:32.405Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2000	2024-12-20T23:02:32.405Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2001	2024-12-20T23:02:32.405Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2002	2024-12-20T23:02:32.429Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
2003	2024-12-20T23:02:32.429Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
2004	2024-12-20T23:02:32.433Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2005	2024-12-20T23:02:32.433Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2006	2024-12-20T23:02:32.435Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 49.923791ms iteration = 18 status = {"errors":[],"finish_invoked_ok":[]}
2007	2024-12-20T23:02:32.436Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
2008	2024-12-20T23:02:32.437Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
2009	2024-12-20T23:02:32.438Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2010	2024-12-20T23:02:32.438Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 53.187981ms iteration = 18 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
2011	2024-12-20T23:02:32.439Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2012	2024-12-20T23:02:32.439Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 53.989343ms iteration = 18 status = {"errors":[],"garbage_collect_requested":[]}
2013	2024-12-20T23:02:32.439Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2014	2024-12-20T23:02:32.439Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2015	2024-12-20T23:02:32.440Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2016	2024-12-20T23:02:32.440Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
2017	2024-12-20T23:02:32.440Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2018	2024-12-20T23:02:32.440Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
2019	2024-12-20T23:02:32.441Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
2020	2024-12-20T23:02:32.441Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2021	2024-12-20T23:02:32.441Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
2022	2024-12-20T23:02:32.441Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2023	2024-12-20T23:02:32.441Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2024	2024-12-20T23:02:32.441Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2025	2024-12-20T23:02:32.442Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2026	2024-12-20T23:02:32.442Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2027	2024-12-20T23:02:32.442Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
2028	2024-12-20T23:02:32.442Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 57.413006ms iteration = 18 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
2029	2024-12-20T23:02:32.445Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2030	2024-12-20T23:02:32.445Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2031	2024-12-20T23:02:32.445Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1167, tv_nsec: 835112766 } })
2032	2024-12-20T23:02:32.445Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
2033	2024-12-20T23:02:32.447Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2034	2024-12-20T23:02:32.450Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 287d6774-d3e6-4b73-8c77-c90534d4a54d, lookup_type: ById(287d6774-d3e6-4b73-8c77-c90534d4a54d) } result = Ok(())
2035	2024-12-20T23:02:32.450Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2036	2024-12-20T23:02:32.450Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2037	2024-12-20T23:02:32.450Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2038	2024-12-20T23:02:32.451Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2039	2024-12-20T23:02:32.453Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 67.970646ms iteration = 18 status = 0
2040	2024-12-20T23:02:32.453Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2041	2024-12-20T23:02:32.454Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2042	2024-12-20T23:02:32.460Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
2043	2024-12-20T23:02:32.460Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 75.086242ms iteration = 52 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
2044	2024-12-20T23:02:32.467Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 81.597931ms iteration = 18 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
2045	2024-12-20T23:02:32.467Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 82.008902ms iteration = 52 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
2046	2024-12-20T23:02:32.468Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
2047	2024-12-20T23:02:32.468Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2048	2024-12-20T23:02:32.468Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f4438e6d-cfdf-4b15-b7e0-e51a369765f9", "content-length": "26", "date": "Fri, 20 Dec 2024 23:02:32 GMT"} })
2049	2024-12-20T23:02:32.468Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2050	2024-12-20T23:02:32.470Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
2051	2024-12-20T23:02:32.470Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2052	2024-12-20T23:02:32.470Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca89adf0-c8e8-4e9a-8e7e-2898139e3920", "content-length": "26", "date": "Fri, 20 Dec 2024 23:02:32 GMT"} })
2053	2024-12-20T23:02:32.470Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2054	2024-12-20T23:02:32.470Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
2055	2024-12-20T23:02:32.471Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
2056	2024-12-20T23:02:32.479Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2057	2024-12-20T23:02:32.479Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
2058	2024-12-20T23:02:32.479Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 94.131715ms iteration = 18 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
2059	2024-12-20T23:02:32.482Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 97.271831ms iteration = 18 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
2060	2024-12-20T23:02:32.503Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 118.134524ms iteration = 19 status = {"physical_disks_added":0}
2061	2024-12-20T23:02:32.541Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2062	2024-12-20T23:02:32.541Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2063	2024-12-20T23:02:32.543Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2064	2024-12-20T23:02:32.544Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
2065	2024-12-20T23:02:32.544Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2066	2024-12-20T23:02:32.545Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2067	2024-12-20T23:02:32.546Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2068	2024-12-20T23:02:32.548Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
2069	2024-12-20T23:02:32.548Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2070	2024-12-20T23:02:32.548Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2071	2024-12-20T23:02:32.552Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 zone_address = fd00:1122:3344:101::2
2072	2024-12-20T23:02:32.552Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2073	2024-12-20T23:02:32.552Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9d5d9409-e120-4d87-b815-269a2e376a63", "content-length": "26", "date": "Fri, 20 Dec 2024 23:02:32 GMT"} })
2074	2024-12-20T23:02:32.552Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2075	2024-12-20T23:02:32.554Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
2076	2024-12-20T23:02:32.576Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 191.311343ms iteration = 18 status = {}
2077	2024-12-20T23:02:32.580Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
2078	2024-12-20T23:02:32.580Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "c98a8563-b729-4017-8f05-ae0dee08c7c8", "content-length": "1", "date": "Fri, 20 Dec 2024 23:02:32 GMT"} })
2079	2024-12-20T23:02:32.582Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
2080	2024-12-20T23:02:32.582Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
2081	2024-12-20T23:02:32.584Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 198.383628ms iteration = 18 status = null
2082	2024-12-20T23:02:32.584Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
2083	2024-12-20T23:02:32.584Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
2084	2024-12-20T23:02:32.585Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd364cb1-ec45-46f4-a621-3f175782ec6c", "content-length": "372", "date": "Fri, 20 Dec 2024 23:02:32 GMT"} })
2085	2024-12-20T23:02:32.585Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2086	2024-12-20T23:02:32.585Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2087	2024-12-20T23:02:32.599Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
2088	2024-12-20T23:02:32.599Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
2089	2024-12-20T23:02:32.599Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "bd03d842-eae8-4c7f-83df-e7335746ae18", "date": "Fri, 20 Dec 2024 23:02:32 GMT"} })
2090	2024-12-20T23:02:32.599Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2091	2024-12-20T23:02:32.600Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2092	2024-12-20T23:02:32.602Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2093	2024-12-20T23:02:32.602Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2094	2024-12-20T23:02:32.605Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2095	2024-12-20T23:02:32.605Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2096	2024-12-20T23:02:32.608Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
2097	2024-12-20T23:02:32.608Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
2098	2024-12-20T23:02:32.608Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "c57f0581-18ec-44db-975a-d1cbc1b13d66", "date": "Fri, 20 Dec 2024 23:02:32 GMT"} })
2099	2024-12-20T23:02:32.609Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 224.115864ms iteration = 18 status = {}
2100	2024-12-20T23:02:32.669Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
2101	2024-12-20T23:02:32.669Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "155c6141-90b0-4f59-bce3-3ecd9f2f3515", "content-length": "691", "date": "Fri, 20 Dec 2024 23:02:32 GMT"} })
2102	2024-12-20T23:02:32.669Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
2103	2024-12-20T23:02:32.669Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
2104	2024-12-20T23:02:32.669Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
2105	2024-12-20T23:02:32.671Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 285.627843ms iteration = 18 status = {}
2106	2024-12-20T23:02:32.695Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2107	2024-12-20T23:02:32.695Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2108	2024-12-20T23:02:32.697Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 03dd12a0-dc2d-46e1-a7f4-d3397b9b7aa7, rack_id: b394d0c6-19a8-4136-9148-af6c47c1f8e0, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(560ca89c-7c0c-4921-8e5c-b57676f695b0) } rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
2109	2024-12-20T23:02:32.697Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2110	2024-12-20T23:02:32.697Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2111	2024-12-20T23:02:32.707Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
2112	2024-12-20T23:02:32.707Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "161f6fb2-d6f7-402d-b549-298bcf26828d", "content-length": "134", "date": "Fri, 20 Dec 2024 23:02:32 GMT"} })
2113	2024-12-20T23:02:32.708Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
2114	2024-12-20T23:02:32.708Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 560ca89c-7c0c-4921-8e5c-b57676f695b0
2115	2024-12-20T23:02:32.708Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
2116	2024-12-20T23:02:32.708Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
2117	2024-12-20T23:02:32.708Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2118	2024-12-20T23:02:32.708Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2119	2024-12-20T23:02:32.708Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2120	2024-12-20T23:02:32.708Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
2121	2024-12-20T23:02:32.708Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2122	2024-12-20T23:02:32.708Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
2123	2024-12-20T23:02:32.708Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
2124	2024-12-20T23:02:32.708Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2125	2024-12-20T23:02:32.709Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2126	2024-12-20T23:02:32.710Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
2127	2024-12-20T23:02:32.710Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "da35eb07-de7b-437c-9e90-c89f9c25fd23", "content-length": "2", "date": "Fri, 20 Dec 2024 23:02:32 GMT"} })
2128	2024-12-20T23:02:32.710Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
2129	2024-12-20T23:02:32.710Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "1ca98166-de6d-468a-a511-e28bbf3fe000", "content-length": "52", "date": "Fri, 20 Dec 2024 23:02:32 GMT"} })
2130	2024-12-20T23:02:32.710Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
2131	2024-12-20T23:02:32.710Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
2132	2024-12-20T23:02:32.835Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "1a269bde-96fb-4a2b-977c-da6f943a39ae", "date": "Fri, 20 Dec 2024 23:02:32 GMT"} })
2133	2024-12-20T23:02:32.835Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch = Switch0
2134	2024-12-20T23:02:32.835Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
2135	2024-12-20T23:02:32.835Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "b7c44660-b382-4407-bc1d-9336d89f95bf", "content-length": "682", "date": "Fri, 20 Dec 2024 23:02:32 GMT"} })
2136	2024-12-20T23:02:32.835Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2137	2024-12-20T23:02:32.836Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2138	2024-12-20T23:02:32.838Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2139	2024-12-20T23:02:32.839Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2140	2024-12-20T23:02:32.842Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2141	2024-12-20T23:02:32.843Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2142	2024-12-20T23:02:32.844Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2143	2024-12-20T23:02:32.845Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2144	2024-12-20T23:02:32.851Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 465.759629ms iteration = 18 status = {}
2145	2024-12-20T23:02:33.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 277 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = f502aaf1-bcac-4f24-b2b5-bfe278b67c47 response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
2146	2024-12-20T23:02:33.167Z	DEBG	nexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
2147	2024-12-20T23:02:33.167Z	DEBG	nexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
2148	2024-12-20T23:02:33.198Z	DEBG	nexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
2149	2024-12-20T23:02:33.198Z	INFO	nexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:35827
2150	2024-12-20T23:02:33.198Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:35827 req_id = d097b8fe-cc0c-46eb-a043-e8facf19208a roles = RoleSet { roles: {} } uri = /metrics/producers
2151	2024-12-20T23:02:33.199Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:35827 req_id = d097b8fe-cc0c-46eb-a043-e8facf19208a resource = Database result = Ok(()) uri = /metrics/producers
2152	2024-12-20T23:02:33.239Z	INFO	nexus: assigned collector to new producer collector_id = 103fc383-10f6-4327-a1b1-dabac4e1e365 file = nexus/src/app/oximeter.rs:76 producer_id = f3b60bd3-6e34-4140-a755-57f26dbc5b34
2153	2024-12-20T23:02:33.239Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 41026 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:35827 req_id = d097b8fe-cc0c-46eb-a043-e8facf19208a response_code = 201 uri = /metrics/producers
2154	2024-12-20T23:02:33.240Z	DEBG	nexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "d097b8fe-cc0c-46eb-a043-e8facf19208a", "content-length": "41", "date": "Fri, 20 Dec 2024 23:02:33 GMT"} })
2155	2024-12-20T23:02:33.241Z	DEBG	nexus (producer-registration-task): registered with nexus successfully
2156	2024-12-20T23:02:33.241Z	DEBG	nexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
2157	2024-12-20T23:02:42.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 53 reason = Timeout
2158	2024-12-20T23:02:42.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 53 reason = Timeout
2159	2024-12-20T23:02:42.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2160	2024-12-20T23:02:42.385Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2161	2024-12-20T23:02:42.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2162	2024-12-20T23:02:42.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2163	2024-12-20T23:02:42.387Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2164	2024-12-20T23:02:42.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2165	2024-12-20T23:02:42.390Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2166	2024-12-20T23:02:42.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2167	2024-12-20T23:02:42.391Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2168	2024-12-20T23:02:42.391Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2169	2024-12-20T23:02:42.393Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2170	2024-12-20T23:02:42.395Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
2171	2024-12-20T23:02:42.395Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2172	2024-12-20T23:02:42.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2173	2024-12-20T23:02:42.409Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
2174	2024-12-20T23:02:42.409Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.963748ms iteration = 53 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
2175	2024-12-20T23:02:42.421Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2176	2024-12-20T23:02:42.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2177	2024-12-20T23:02:42.423Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 37.862439ms iteration = 53 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
2178	2024-12-20T23:02:43.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 303 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = bd5ec0df-e8db-4d68-8a89-cd68f6c2b392 response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
2179	2024-12-20T23:02:52.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 54 reason = Timeout
2180	2024-12-20T23:02:52.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2181	2024-12-20T23:02:52.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 54 reason = Timeout
2182	2024-12-20T23:02:52.385Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2183	2024-12-20T23:02:52.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2184	2024-12-20T23:02:52.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2185	2024-12-20T23:02:52.387Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2186	2024-12-20T23:02:52.387Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2187	2024-12-20T23:02:52.387Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2188	2024-12-20T23:02:52.388Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.873586ms iteration = 54 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
2189	2024-12-20T23:02:52.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2190	2024-12-20T23:02:52.389Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2191	2024-12-20T23:02:52.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2192	2024-12-20T23:02:52.391Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2193	2024-12-20T23:02:52.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2194	2024-12-20T23:02:52.393Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2195	2024-12-20T23:02:52.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
2196	2024-12-20T23:02:52.396Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2197	2024-12-20T23:02:52.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2198	2024-12-20T23:02:52.405Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
2199	2024-12-20T23:02:52.405Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.40911ms iteration = 54 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
2200	2024-12-20T23:02:53.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 300 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 5a837de8-e4f2-4816-b0a0-84be635712aa response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
2201	2024-12-20T23:03:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 19 reason = Timeout
2202	2024-12-20T23:03:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
2203	2024-12-20T23:03:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 19 reason = Timeout
2204	2024-12-20T23:03:02.385Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
2205	2024-12-20T23:03:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 19 reason = Timeout
2206	2024-12-20T23:03:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2207	2024-12-20T23:03:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
2208	2024-12-20T23:03:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = external_endpoints iteration = 10 reason = Timeout
2209	2024-12-20T23:03:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 10 reason = Timeout
2210	2024-12-20T23:03:02.385Z	INFO	nexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
2211	2024-12-20T23:03:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2212	2024-12-20T23:03:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
2213	2024-12-20T23:03:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 10 reason = Timeout
2214	2024-12-20T23:03:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 5 reason = Timeout
2215	2024-12-20T23:03:02.385Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
2216	2024-12-20T23:03:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 20 reason = Timeout
2217	2024-12-20T23:03:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2218	2024-12-20T23:03:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2219	2024-12-20T23:03:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2220	2024-12-20T23:03:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 55 reason = Timeout
2221	2024-12-20T23:03:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 19 reason = Timeout
2222	2024-12-20T23:03:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
2223	2024-12-20T23:03:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2224	2024-12-20T23:03:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 12 reason = Timeout
2225	2024-12-20T23:03:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 19 reason = Timeout
2226	2024-12-20T23:03:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
2227	2024-12-20T23:03:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 19 reason = Timeout
2228	2024-12-20T23:03:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 19 reason = Timeout
2229	2024-12-20T23:03:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2230	2024-12-20T23:03:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
2231	2024-12-20T23:03:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 19 reason = Timeout
2232	2024-12-20T23:03:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2233	2024-12-20T23:03:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 10 reason = Timeout
2234	2024-12-20T23:03:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 19 reason = Timeout
2235	2024-12-20T23:03:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2236	2024-12-20T23:03:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 11 reason = Timeout
2237	2024-12-20T23:03:02.385Z	WARN	nexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7
2238	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 13.65\u{b5}s iteration = 11 status = {"enabled":false,"target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7"}
2239	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 10 reason = Timeout
2240	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 19 reason = Timeout
2241	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
2242	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2243	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 19 reason = Timeout
2244	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
2245	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 10 reason = Timeout
2246	2024-12-20T23:03:02.386Z	INFO	nexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-20 22:53:02.385789180 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
2247	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 19 reason = Timeout
2248	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
2249	2024-12-20T23:03:02.386Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
2250	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2251	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 10 reason = Timeout
2252	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 12 reason = Timeout
2253	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
2254	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2255	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 19 reason = Timeout
2256	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 10 reason = Timeout
2257	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
2258	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 19 reason = Timeout
2259	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 55 reason = Timeout
2260	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
2261	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2262	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 10 reason = Timeout
2263	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
2264	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 10 reason = Timeout
2265	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
2266	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
2267	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2268	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
2269	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2270	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2271	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
2272	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
2273	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2274	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2275	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
2276	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
2277	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2278	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
2279	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
2280	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2281	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2282	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2283	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
2284	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2285	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
2286	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2287	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
2288	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2289	2024-12-20T23:03:02.386Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1e296b8d-6800-4056-8034-a1ec2d005fff.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("ace0592c-7755-46f8-b47a-c9b6444fe64c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("ace0592c-7755-46f8-b47a-c9b6444fe64c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1197, tv_nsec: 776114786 } })
2290	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2291	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
2292	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
2293	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
2294	2024-12-20T23:03:02.386Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("616ce87c-1f39-49d9-ba35-ee63b2650ecd.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b4615881-2f68-4c82-be17-071bf3ae3641.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c9217e92-0af3-4930-acb2-aa82ded5b782.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("c9217e92-0af3-4930-acb2-aa82ded5b782.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 1197, tv_nsec: 776193588 } })
2295	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
2296	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2297	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
2298	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2299	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
2300	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
2301	2024-12-20T23:03:02.386Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.064659ms iteration = 10 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
2302	2024-12-20T23:03:02.387Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
2303	2024-12-20T23:03:02.387Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.328856ms iteration = 10 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
2304	2024-12-20T23:03:02.387Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.106808ms iteration = 19 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
2305	2024-12-20T23:03:02.388Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2306	2024-12-20T23:03:02.388Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1197, tv_nsec: 778339767 } })
2307	2024-12-20T23:03:02.388Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
2308	2024-12-20T23:03:02.389Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2309	2024-12-20T23:03:02.389Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2310	2024-12-20T23:03:02.390Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2311	2024-12-20T23:03:02.391Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
2312	2024-12-20T23:03:02.391Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2313	2024-12-20T23:03:02.391Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
2314	2024-12-20T23:03:02.391Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
2315	2024-12-20T23:03:02.391Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
2316	2024-12-20T23:03:02.392Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
2317	2024-12-20T23:03:02.392Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2318	2024-12-20T23:03:02.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
2319	2024-12-20T23:03:02.392Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2320	2024-12-20T23:03:02.392Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
2321	2024-12-20T23:03:02.392Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2322	2024-12-20T23:03:02.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2323	2024-12-20T23:03:02.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
2324	2024-12-20T23:03:02.392Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2325	2024-12-20T23:03:02.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2326	2024-12-20T23:03:02.393Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2327	2024-12-20T23:03:02.393Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2328	2024-12-20T23:03:02.393Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2329	2024-12-20T23:03:02.395Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1197, tv_nsec: 784812104 } })
2330	2024-12-20T23:03:02.395Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2331	2024-12-20T23:03:02.395Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
2332	2024-12-20T23:03:02.396Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2333	2024-12-20T23:03:02.396Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2334	2024-12-20T23:03:02.397Z	DEBG	nexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
2335	2024-12-20T23:03:02.397Z	INFO	nexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
2336	2024-12-20T23:03:02.397Z	DEBG	nexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 11.888317ms iteration = 10 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
2337	2024-12-20T23:03:02.398Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
2338	2024-12-20T23:03:02.398Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2339	2024-12-20T23:03:02.398Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2340	2024-12-20T23:03:02.399Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
2341	2024-12-20T23:03:02.399Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2342	2024-12-20T23:03:02.399Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2343	2024-12-20T23:03:02.399Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
2344	2024-12-20T23:03:02.399Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2345	2024-12-20T23:03:02.399Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2346	2024-12-20T23:03:02.401Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2347	2024-12-20T23:03:02.402Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2348	2024-12-20T23:03:02.406Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2349	2024-12-20T23:03:02.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2350	2024-12-20T23:03:02.408Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2351	2024-12-20T23:03:02.409Z	DEBG	nexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 23.350292ms iteration = 10 status = {"errors":[],"found_port_ok":[]}
2352	2024-12-20T23:03:02.409Z	INFO	nexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-20 22:53:02.385789180 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
2353	2024-12-20T23:03:02.409Z	DEBG	nexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 24.03432ms iteration = 10 status = {"expiration":"2024-12-20T22:53:02.385789180Z","pruned":[]}
2354	2024-12-20T23:03:02.409Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2355	2024-12-20T23:03:02.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
2356	2024-12-20T23:03:02.412Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
2357	2024-12-20T23:03:02.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
2358	2024-12-20T23:03:02.412Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2359	2024-12-20T23:03:02.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
2360	2024-12-20T23:03:02.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
2361	2024-12-20T23:03:02.414Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2362	2024-12-20T23:03:02.417Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2363	2024-12-20T23:03:02.417Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2364	2024-12-20T23:03:02.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2365	2024-12-20T23:03:02.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
2366	2024-12-20T23:03:02.417Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2367	2024-12-20T23:03:02.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
2368	2024-12-20T23:03:02.418Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2369	2024-12-20T23:03:02.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2370	2024-12-20T23:03:02.420Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2371	2024-12-20T23:03:02.420Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1197, tv_nsec: 810250863 } })
2372	2024-12-20T23:03:02.420Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
2373	2024-12-20T23:03:02.421Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2374	2024-12-20T23:03:02.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2375	2024-12-20T23:03:02.421Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2376	2024-12-20T23:03:02.425Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2377	2024-12-20T23:03:02.428Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2378	2024-12-20T23:03:02.431Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
2379	2024-12-20T23:03:02.431Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2380	2024-12-20T23:03:02.431Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2381	2024-12-20T23:03:02.433Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
2382	2024-12-20T23:03:02.433Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
2383	2024-12-20T23:03:02.437Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 51.539435ms iteration = 19 status = {"errors":[],"finish_invoked_ok":[]}
2384	2024-12-20T23:03:02.437Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 52.232175ms iteration = 19 status = {"errors":[],"garbage_collect_requested":[]}
2385	2024-12-20T23:03:02.448Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 63.274367ms iteration = 19 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
2386	2024-12-20T23:03:02.449Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
2387	2024-12-20T23:03:02.449Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
2388	2024-12-20T23:03:02.451Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 65.790226ms iteration = 19 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
2389	2024-12-20T23:03:02.452Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2390	2024-12-20T23:03:02.454Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2391	2024-12-20T23:03:02.454Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2392	2024-12-20T23:03:02.456Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2393	2024-12-20T23:03:02.456Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2394	2024-12-20T23:03:02.456Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2395	2024-12-20T23:03:02.457Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2396	2024-12-20T23:03:02.459Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2397	2024-12-20T23:03:02.459Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
2398	2024-12-20T23:03:02.459Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2399	2024-12-20T23:03:02.459Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
2400	2024-12-20T23:03:02.459Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2401	2024-12-20T23:03:02.459Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 73.887479ms iteration = 55 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
2402	2024-12-20T23:03:02.460Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2403	2024-12-20T23:03:02.461Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
2404	2024-12-20T23:03:02.461Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2405	2024-12-20T23:03:02.462Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
2406	2024-12-20T23:03:02.462Z	DEBG	nexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
2407	2024-12-20T23:03:02.462Z	INFO	nexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
2408	2024-12-20T23:03:02.462Z	DEBG	nexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 77.370105ms iteration = 10 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
2409	2024-12-20T23:03:02.463Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2410	2024-12-20T23:03:02.463Z	DEBG	nexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 78.167626ms iteration = 5 status = {"nsuccess":5}
2411	2024-12-20T23:03:02.465Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
2412	2024-12-20T23:03:02.465Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2413	2024-12-20T23:03:02.466Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2414	2024-12-20T23:03:02.466Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
2415	2024-12-20T23:03:02.466Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
2416	2024-12-20T23:03:02.466Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
2417	2024-12-20T23:03:02.466Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2418	2024-12-20T23:03:02.466Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2419	2024-12-20T23:03:02.467Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2420	2024-12-20T23:03:02.468Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2421	2024-12-20T23:03:02.468Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2422	2024-12-20T23:03:02.468Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
2423	2024-12-20T23:03:02.468Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2424	2024-12-20T23:03:02.468Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
2425	2024-12-20T23:03:02.470Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
2426	2024-12-20T23:03:02.470Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2427	2024-12-20T23:03:02.471Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2428	2024-12-20T23:03:02.471Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2429	2024-12-20T23:03:02.471Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2430	2024-12-20T23:03:02.471Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2431	2024-12-20T23:03:02.472Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2432	2024-12-20T23:03:02.472Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2433	2024-12-20T23:03:02.473Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
2434	2024-12-20T23:03:02.473Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 87.180064ms iteration = 19 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
2435	2024-12-20T23:03:02.474Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2436	2024-12-20T23:03:02.475Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2437	2024-12-20T23:03:02.476Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2438	2024-12-20T23:03:02.477Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2439	2024-12-20T23:03:02.479Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
2440	2024-12-20T23:03:02.479Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 93.534178ms iteration = 55 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
2441	2024-12-20T23:03:02.479Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2442	2024-12-20T23:03:02.479Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2443	2024-12-20T23:03:02.480Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2444	2024-12-20T23:03:02.480Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 287d6774-d3e6-4b73-8c77-c90534d4a54d, lookup_type: ById(287d6774-d3e6-4b73-8c77-c90534d4a54d) } result = Ok(())
2445	2024-12-20T23:03:02.480Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2446	2024-12-20T23:03:02.480Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2447	2024-12-20T23:03:02.482Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
2448	2024-12-20T23:03:02.482Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
2449	2024-12-20T23:03:02.482Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
2450	2024-12-20T23:03:02.482Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2451	2024-12-20T23:03:02.482Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 96.934252ms iteration = 19 status = 0
2452	2024-12-20T23:03:02.482Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2453	2024-12-20T23:03:02.487Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2454	2024-12-20T23:03:02.487Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2455	2024-12-20T23:03:02.489Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
2456	2024-12-20T23:03:02.489Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
2457	2024-12-20T23:03:02.493Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2458	2024-12-20T23:03:02.494Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2459	2024-12-20T23:03:02.499Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2460	2024-12-20T23:03:02.500Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
2461	2024-12-20T23:03:02.500Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2462	2024-12-20T23:03:02.501Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2463	2024-12-20T23:03:02.507Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2464	2024-12-20T23:03:02.507Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
2465	2024-12-20T23:03:02.507Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 121.949589ms iteration = 19 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
2466	2024-12-20T23:03:02.513Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
2467	2024-12-20T23:03:02.516Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
2468	2024-12-20T23:03:02.519Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
2469	2024-12-20T23:03:02.519Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 5fa20c62-75c0-4531-be07-51b87d303d18 dns_zone_name = oxide.test found_so_far = 0 version = 2
2470	2024-12-20T23:03:02.519Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2471	2024-12-20T23:03:02.520Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2472	2024-12-20T23:03:02.522Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2473	2024-12-20T23:03:02.524Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2474	2024-12-20T23:03:02.526Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 141.055233ms iteration = 20 status = {"physical_disks_added":0}
2475	2024-12-20T23:03:02.527Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2476	2024-12-20T23:03:02.527Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2477	2024-12-20T23:03:02.527Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2478	2024-12-20T23:03:02.527Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2479	2024-12-20T23:03:02.527Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2480	2024-12-20T23:03:02.527Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2481	2024-12-20T23:03:02.528Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
2482	2024-12-20T23:03:02.528Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2483	2024-12-20T23:03:02.529Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "df6a536f-ac2f-415b-8a92-6780e0bbe973", "content-length": "26", "date": "Fri, 20 Dec 2024 23:03:02 GMT"} })
2484	2024-12-20T23:03:02.529Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2485	2024-12-20T23:03:02.529Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
2486	2024-12-20T23:03:02.529Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 1e86bf23-6ffd-4a12-93c7-0061d7d17590 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
2487	2024-12-20T23:03:02.529Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2488	2024-12-20T23:03:02.529Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2489	2024-12-20T23:03:02.530Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 5fa20c62-75c0-4531-be07-51b87d303d18 dns_zone_name = oxide.test found_so_far = 1 version = 2
2490	2024-12-20T23:03:02.530Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
2491	2024-12-20T23:03:02.530Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-20 22:54:39.899407 UTC dns_group = external generation = 2
2492	2024-12-20T23:03:02.530Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 144.94235ms iteration = 10 status = {"generation":2}
2493	2024-12-20T23:03:02.532Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
2494	2024-12-20T23:03:02.532Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 147.069848ms iteration = 19 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
2495	2024-12-20T23:03:02.532Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2496	2024-12-20T23:03:02.535Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2497	2024-12-20T23:03:02.535Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2498	2024-12-20T23:03:02.535Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2499	2024-12-20T23:03:02.538Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 1e86bf23-6ffd-4a12-93c7-0061d7d17590 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
2500	2024-12-20T23:03:02.538Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
2501	2024-12-20T23:03:02.538Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-20 22:54:38.956071 UTC dns_group = internal generation = 1
2502	2024-12-20T23:03:02.538Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 152.310492ms iteration = 10 status = {"generation":1}
2503	2024-12-20T23:03:02.540Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2504	2024-12-20T23:03:02.541Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2505	2024-12-20T23:03:02.543Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
2506	2024-12-20T23:03:02.543Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2507	2024-12-20T23:03:02.543Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "52b224df-8cc6-4c1c-b189-b9b426a4fb28", "content-length": "26", "date": "Fri, 20 Dec 2024 23:03:02 GMT"} })
2508	2024-12-20T23:03:02.543Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2509	2024-12-20T23:03:02.545Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
2510	2024-12-20T23:03:02.559Z	DEBG	nexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-20T22:53:58.073373Z, time_modified: 2024-12-20T22:53:58.073373Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, db_silo: Silo { identity: SiloIdentity { id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-20T22:54:39.847015Z, time_modified: 2024-12-20T22:54:39.847015Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, db_silo: Silo { identity: SiloIdentity { id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-20T22:54:39.847015Z, time_modified: 2024-12-20T22:54:39.847015Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156" }] }) }
2511	2024-12-20T23:03:02.559Z	DEBG	nexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 174.367828ms iteration = 10 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"ee579470-b16b-4af1-ad3a-b3d29f9a2fb2","tls_certs":["ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156"]}},"default_endpoint":{"silo_id":"ee579470-b16b-4af1-ad3a-b3d29f9a2fb2","tls_certs":["ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
2512	2024-12-20T23:03:02.563Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 zone_address = fd00:1122:3344:101::2
2513	2024-12-20T23:03:02.563Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2514	2024-12-20T23:03:02.563Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "226f827e-f334-40bd-8dd4-816f2ccbc9d7", "content-length": "26", "date": "Fri, 20 Dec 2024 23:03:02 GMT"} })
2515	2024-12-20T23:03:02.563Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2516	2024-12-20T23:03:02.566Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
2517	2024-12-20T23:03:02.586Z	WARN	nexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
2518	2024-12-20T23:03:02.586Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 200.814105ms iteration = 10 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
2519	2024-12-20T23:03:02.611Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2520	2024-12-20T23:03:02.612Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2521	2024-12-20T23:03:02.613Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2522	2024-12-20T23:03:02.616Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
2523	2024-12-20T23:03:02.616Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2524	2024-12-20T23:03:02.616Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2525	2024-12-20T23:03:02.700Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
2526	2024-12-20T23:03:02.701Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad7bf10e-0bfa-486c-b745-80d4832bb234", "content-length": "1", "date": "Fri, 20 Dec 2024 23:03:02 GMT"} })
2527	2024-12-20T23:03:02.702Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
2528	2024-12-20T23:03:02.702Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
2529	2024-12-20T23:03:02.702Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
2530	2024-12-20T23:03:02.703Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
2531	2024-12-20T23:03:02.703Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f6687b3-5381-41af-a277-274b1c583b4c", "content-length": "372", "date": "Fri, 20 Dec 2024 23:03:02 GMT"} })
2532	2024-12-20T23:03:02.703Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2533	2024-12-20T23:03:02.703Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2534	2024-12-20T23:03:02.704Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2535	2024-12-20T23:03:02.705Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 319.660907ms iteration = 19 status = null
2536	2024-12-20T23:03:02.706Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
2537	2024-12-20T23:03:02.706Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
2538	2024-12-20T23:03:02.706Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2539	2024-12-20T23:03:02.706Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "1aa3d8c0-7b22-4b4d-a03d-df88005be6f8", "date": "Fri, 20 Dec 2024 23:03:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2540	2024-12-20T23:03:02.706Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2541	2024-12-20T23:03:02.706Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2542	2024-12-20T23:03:02.708Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "d17c1023-0c5a-421c-9646-f26ab1bf3310", "date": "Fri, 20 Dec 2024 23:03:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2543	2024-12-20T23:03:02.708Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2544	2024-12-20T23:03:02.708Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
2545	2024-12-20T23:03:02.710Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2546	2024-12-20T23:03:02.710Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 325.116288ms iteration = 12 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
2547	2024-12-20T23:03:02.768Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
2548	2024-12-20T23:03:02.768Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
2549	2024-12-20T23:03:02.768Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "5a18cbc3-3288-4411-93a8-c102d1be4735", "date": "Fri, 20 Dec 2024 23:03:02 GMT"} })
2550	2024-12-20T23:03:02.768Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2551	2024-12-20T23:03:02.768Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2552	2024-12-20T23:03:02.772Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2553	2024-12-20T23:03:02.773Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2554	2024-12-20T23:03:02.781Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2555	2024-12-20T23:03:02.781Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2556	2024-12-20T23:03:02.795Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
2557	2024-12-20T23:03:02.795Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
2558	2024-12-20T23:03:02.796Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "984620cf-baf4-4063-aad2-2a60bd3fa1f1", "date": "Fri, 20 Dec 2024 23:03:02 GMT"} })
2559	2024-12-20T23:03:02.798Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 412.915548ms iteration = 19 status = {}
2560	2024-12-20T23:03:02.819Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 433.998586ms iteration = 19 status = {}
2561	2024-12-20T23:03:02.850Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
2562	2024-12-20T23:03:02.851Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "33a4f4f9-1eb1-487d-9996-17677de9c2bb", "content-length": "691", "date": "Fri, 20 Dec 2024 23:03:02 GMT"} })
2563	2024-12-20T23:03:02.851Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
2564	2024-12-20T23:03:02.851Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
2565	2024-12-20T23:03:02.851Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
2566	2024-12-20T23:03:02.852Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 467.009924ms iteration = 19 status = {}
2567	2024-12-20T23:03:02.853Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
2568	2024-12-20T23:03:02.853Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "c24d8b38-a638-4934-b6da-90a27bad80e1", "date": "Fri, 20 Dec 2024 23:03:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
2569	2024-12-20T23:03:02.853Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
2570	2024-12-20T23:03:02.855Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "41562e9c-26e2-478c-a5bd-79ecb6539c48", "date": "Fri, 20 Dec 2024 23:03:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
2571	2024-12-20T23:03:02.855Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
2572	2024-12-20T23:03:02.857Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "c9c417c2-d498-4a43-aee3-a9a930277da4", "date": "Fri, 20 Dec 2024 23:03:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
2573	2024-12-20T23:03:02.857Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
2574	2024-12-20T23:03:02.859Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
2575	2024-12-20T23:03:02.859Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 474.1396ms iteration = 12 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
2576	2024-12-20T23:03:02.916Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2577	2024-12-20T23:03:02.916Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2578	2024-12-20T23:03:02.917Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 03dd12a0-dc2d-46e1-a7f4-d3397b9b7aa7, rack_id: b394d0c6-19a8-4136-9148-af6c47c1f8e0, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(560ca89c-7c0c-4921-8e5c-b57676f695b0) } rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
2579	2024-12-20T23:03:02.917Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2580	2024-12-20T23:03:02.918Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2581	2024-12-20T23:03:02.928Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
2582	2024-12-20T23:03:02.928Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "19093355-732c-4695-8eda-190e3665a52c", "content-length": "134", "date": "Fri, 20 Dec 2024 23:03:02 GMT"} })
2583	2024-12-20T23:03:02.928Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
2584	2024-12-20T23:03:02.928Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 560ca89c-7c0c-4921-8e5c-b57676f695b0
2585	2024-12-20T23:03:02.928Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
2586	2024-12-20T23:03:02.928Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
2587	2024-12-20T23:03:02.928Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2588	2024-12-20T23:03:02.928Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2589	2024-12-20T23:03:02.928Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2590	2024-12-20T23:03:02.928Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
2591	2024-12-20T23:03:02.929Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2592	2024-12-20T23:03:02.929Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
2593	2024-12-20T23:03:02.929Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
2594	2024-12-20T23:03:02.929Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2595	2024-12-20T23:03:02.929Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2596	2024-12-20T23:03:02.931Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
2597	2024-12-20T23:03:02.932Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "36b65210-8d0e-4af3-8697-6cbc21f6a368", "content-length": "2", "date": "Fri, 20 Dec 2024 23:03:02 GMT"} })
2598	2024-12-20T23:03:02.932Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
2599	2024-12-20T23:03:02.932Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "272b12b7-6d50-497c-8219-6d166e905bd0", "content-length": "52", "date": "Fri, 20 Dec 2024 23:03:02 GMT"} })
2600	2024-12-20T23:03:02.932Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
2601	2024-12-20T23:03:02.932Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
2602	2024-12-20T23:03:03.026Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "4a08801f-aafb-4451-979b-22f14e41febb", "date": "Fri, 20 Dec 2024 23:03:02 GMT"} })
2603	2024-12-20T23:03:03.026Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch = Switch0
2604	2024-12-20T23:03:03.026Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
2605	2024-12-20T23:03:03.027Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "95a9f289-d12b-4223-b9c9-c7e652993bc2", "content-length": "682", "date": "Fri, 20 Dec 2024 23:03:02 GMT"} })
2606	2024-12-20T23:03:03.027Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2607	2024-12-20T23:03:03.027Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2608	2024-12-20T23:03:03.031Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2609	2024-12-20T23:03:03.031Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2610	2024-12-20T23:03:03.034Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2611	2024-12-20T23:03:03.034Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2612	2024-12-20T23:03:03.036Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2613	2024-12-20T23:03:03.036Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2614	2024-12-20T23:03:03.044Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 658.584874ms iteration = 19 status = {}
2615	2024-12-20T23:03:03.149Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 281 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 0ed02ec4-219a-4eec-a6d4-7cc393e4a25e response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
2616	2024-12-20T23:03:12.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 56 reason = Timeout
2617	2024-12-20T23:03:12.385Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2618	2024-12-20T23:03:12.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 56 reason = Timeout
2619	2024-12-20T23:03:12.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2620	2024-12-20T23:03:12.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2621	2024-12-20T23:03:12.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2622	2024-12-20T23:03:12.387Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2623	2024-12-20T23:03:12.388Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2624	2024-12-20T23:03:12.388Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2625	2024-12-20T23:03:12.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2626	2024-12-20T23:03:12.390Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2627	2024-12-20T23:03:12.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2628	2024-12-20T23:03:12.390Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 5.039068ms iteration = 56 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
2629	2024-12-20T23:03:12.392Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2630	2024-12-20T23:03:12.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2631	2024-12-20T23:03:12.394Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2632	2024-12-20T23:03:12.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
2633	2024-12-20T23:03:12.396Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2634	2024-12-20T23:03:12.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2635	2024-12-20T23:03:12.407Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
2636	2024-12-20T23:03:12.408Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.533029ms iteration = 56 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
2637	2024-12-20T23:03:13.149Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 253 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 1293d441-580a-488d-a0a0-89fe15608789 response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
2638	2024-12-20T23:03:22.384Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 57 reason = Timeout
2639	2024-12-20T23:03:22.384Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2640	2024-12-20T23:03:22.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 57 reason = Timeout
2641	2024-12-20T23:03:22.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2642	2024-12-20T23:03:22.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2643	2024-12-20T23:03:22.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2644	2024-12-20T23:03:22.386Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2645	2024-12-20T23:03:22.387Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2646	2024-12-20T23:03:22.388Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2647	2024-12-20T23:03:22.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2648	2024-12-20T23:03:22.389Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2649	2024-12-20T23:03:22.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2650	2024-12-20T23:03:22.390Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 5.301595ms iteration = 57 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
2651	2024-12-20T23:03:22.391Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2652	2024-12-20T23:03:22.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2653	2024-12-20T23:03:22.393Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2654	2024-12-20T23:03:22.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
2655	2024-12-20T23:03:22.396Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2656	2024-12-20T23:03:22.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2657	2024-12-20T23:03:22.406Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
2658	2024-12-20T23:03:22.406Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.48358ms iteration = 57 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
2659	2024-12-20T23:03:23.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 261 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 89df4509-0791-4913-8b3e-e03bc0b92287 response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
2660	2024-12-20T23:03:28.139Z	INFO	nexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:42603
2661	2024-12-20T23:03:28.139Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42603 req_id = 4c196232-22a3-4d4a-a082-ec93a0653381 roles = RoleSet { roles: {} } uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?sort_by=id_ascending
2662	2024-12-20T23:03:28.139Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42603 req_id = 4c196232-22a3-4d4a-a082-ec93a0653381 resource = Database result = Ok(()) uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?sort_by=id_ascending
2663	2024-12-20T23:03:28.141Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2138 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42603 req_id = 4c196232-22a3-4d4a-a082-ec93a0653381 response_code = 200 uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?sort_by=id_ascending
2664	2024-12-20T23:03:28.141Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42603 req_id = e6ed3b78-a8ba-48d7-bef1-d68300aaa146 roles = RoleSet { roles: {} } uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmM2I2MGJkMy02ZTM0LTQxNDAtYTc1NS01N2YyNmRiYzViMzQifX0%3D
2665	2024-12-20T23:03:28.141Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42603 req_id = e6ed3b78-a8ba-48d7-bef1-d68300aaa146 resource = Database result = Ok(()) uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmM2I2MGJkMy02ZTM0LTQxNDAtYTc1NS01N2YyNmRiYzViMzQifX0%3D
2666	2024-12-20T23:03:28.143Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2453 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42603 req_id = e6ed3b78-a8ba-48d7-bef1-d68300aaa146 response_code = 200 uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmM2I2MGJkMy02ZTM0LTQxNDAtYTc1NS01N2YyNmRiYzViMzQifX0%3D
2667	2024-12-20T23:03:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 20 reason = Timeout
2668	2024-12-20T23:03:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
2669	2024-12-20T23:03:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 20 reason = Timeout
2670	2024-12-20T23:03:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2671	2024-12-20T23:03:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 20 reason = Timeout
2672	2024-12-20T23:03:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
2673	2024-12-20T23:03:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 20 reason = Timeout
2674	2024-12-20T23:03:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 20 reason = Timeout
2675	2024-12-20T23:03:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2676	2024-12-20T23:03:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 20 reason = Timeout
2677	2024-12-20T23:03:32.385Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
2678	2024-12-20T23:03:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
2679	2024-12-20T23:03:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2680	2024-12-20T23:03:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 20 reason = Timeout
2681	2024-12-20T23:03:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
2682	2024-12-20T23:03:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 20 reason = Timeout
2683	2024-12-20T23:03:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 20 reason = Timeout
2684	2024-12-20T23:03:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 20 reason = Timeout
2685	2024-12-20T23:03:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
2686	2024-12-20T23:03:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2687	2024-12-20T23:03:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 20 reason = Timeout
2688	2024-12-20T23:03:32.385Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
2689	2024-12-20T23:03:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 20 reason = Timeout
2690	2024-12-20T23:03:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2691	2024-12-20T23:03:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
2692	2024-12-20T23:03:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 58 reason = Timeout
2693	2024-12-20T23:03:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2694	2024-12-20T23:03:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 21 reason = Timeout
2695	2024-12-20T23:03:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2696	2024-12-20T23:03:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 58 reason = Timeout
2697	2024-12-20T23:03:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 20 reason = Timeout
2698	2024-12-20T23:03:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2699	2024-12-20T23:03:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
2700	2024-12-20T23:03:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 20 reason = Timeout
2701	2024-12-20T23:03:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2702	2024-12-20T23:03:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
2703	2024-12-20T23:03:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
2704	2024-12-20T23:03:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2705	2024-12-20T23:03:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2706	2024-12-20T23:03:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2707	2024-12-20T23:03:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
2708	2024-12-20T23:03:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
2709	2024-12-20T23:03:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
2710	2024-12-20T23:03:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
2711	2024-12-20T23:03:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
2712	2024-12-20T23:03:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2713	2024-12-20T23:03:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2714	2024-12-20T23:03:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2715	2024-12-20T23:03:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2716	2024-12-20T23:03:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
2717	2024-12-20T23:03:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2718	2024-12-20T23:03:32.388Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 2.784646ms iteration = 20 status = {"errors":[],"garbage_collect_requested":[]}
2719	2024-12-20T23:03:32.388Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1227, tv_nsec: 778110130 } })
2720	2024-12-20T23:03:32.388Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
2721	2024-12-20T23:03:32.389Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.843306ms iteration = 20 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
2722	2024-12-20T23:03:32.389Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2723	2024-12-20T23:03:32.390Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2724	2024-12-20T23:03:32.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2725	2024-12-20T23:03:32.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2726	2024-12-20T23:03:32.391Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2727	2024-12-20T23:03:32.391Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2728	2024-12-20T23:03:32.391Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
2729	2024-12-20T23:03:32.391Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2730	2024-12-20T23:03:32.391Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
2731	2024-12-20T23:03:32.392Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1227, tv_nsec: 782272575 } })
2732	2024-12-20T23:03:32.392Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
2733	2024-12-20T23:03:32.393Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2734	2024-12-20T23:03:32.393Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
2735	2024-12-20T23:03:32.393Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
2736	2024-12-20T23:03:32.393Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2737	2024-12-20T23:03:32.393Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
2738	2024-12-20T23:03:32.393Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
2739	2024-12-20T23:03:32.393Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
2740	2024-12-20T23:03:32.395Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
2741	2024-12-20T23:03:32.395Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2742	2024-12-20T23:03:32.395Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2743	2024-12-20T23:03:32.395Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2744	2024-12-20T23:03:32.395Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2745	2024-12-20T23:03:32.395Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 10.318783ms iteration = 20 status = {"errors":[],"finish_invoked_ok":[]}
2746	2024-12-20T23:03:32.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2747	2024-12-20T23:03:32.396Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2748	2024-12-20T23:03:32.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2749	2024-12-20T23:03:32.397Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2750	2024-12-20T23:03:32.398Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2751	2024-12-20T23:03:32.398Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
2752	2024-12-20T23:03:32.398Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2753	2024-12-20T23:03:32.398Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2754	2024-12-20T23:03:32.398Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2755	2024-12-20T23:03:32.398Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2756	2024-12-20T23:03:32.398Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2757	2024-12-20T23:03:32.398Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2758	2024-12-20T23:03:32.399Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2759	2024-12-20T23:03:32.399Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2760	2024-12-20T23:03:32.400Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
2761	2024-12-20T23:03:32.400Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2762	2024-12-20T23:03:32.400Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2763	2024-12-20T23:03:32.401Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2764	2024-12-20T23:03:32.401Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
2765	2024-12-20T23:03:32.401Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2766	2024-12-20T23:03:32.401Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
2767	2024-12-20T23:03:32.401Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 16.045331ms iteration = 20 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
2768	2024-12-20T23:03:32.402Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2769	2024-12-20T23:03:32.402Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2770	2024-12-20T23:03:32.403Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2771	2024-12-20T23:03:32.403Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2772	2024-12-20T23:03:32.403Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2773	2024-12-20T23:03:32.403Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
2774	2024-12-20T23:03:32.403Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2775	2024-12-20T23:03:32.403Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1227, tv_nsec: 793480803 } })
2776	2024-12-20T23:03:32.403Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2777	2024-12-20T23:03:32.404Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
2778	2024-12-20T23:03:32.405Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 20.006029ms iteration = 20 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
2779	2024-12-20T23:03:32.406Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2780	2024-12-20T23:03:32.406Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2781	2024-12-20T23:03:32.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2782	2024-12-20T23:03:32.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2783	2024-12-20T23:03:32.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2784	2024-12-20T23:03:32.408Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2785	2024-12-20T23:03:32.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2786	2024-12-20T23:03:32.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
2787	2024-12-20T23:03:32.411Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2788	2024-12-20T23:03:32.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2789	2024-12-20T23:03:32.425Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
2790	2024-12-20T23:03:32.425Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
2791	2024-12-20T23:03:32.426Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
2792	2024-12-20T23:03:32.426Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 40.740569ms iteration = 58 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
2793	2024-12-20T23:03:32.438Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 52.81337ms iteration = 58 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
2794	2024-12-20T23:03:32.438Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2795	2024-12-20T23:03:32.439Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2796	2024-12-20T23:03:32.439Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2797	2024-12-20T23:03:32.439Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
2798	2024-12-20T23:03:32.439Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 53.550211ms iteration = 20 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
2799	2024-12-20T23:03:32.441Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2800	2024-12-20T23:03:32.442Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2801	2024-12-20T23:03:32.443Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
2802	2024-12-20T23:03:32.443Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 57.92402ms iteration = 20 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
2803	2024-12-20T23:03:32.465Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 80.312905ms iteration = 20 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
2804	2024-12-20T23:03:32.482Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
2805	2024-12-20T23:03:32.482Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2806	2024-12-20T23:03:32.482Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "703cea7c-983e-4257-9898-780189d23179", "content-length": "26", "date": "Fri, 20 Dec 2024 23:03:32 GMT"} })
2807	2024-12-20T23:03:32.482Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2808	2024-12-20T23:03:32.485Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
2809	2024-12-20T23:03:32.499Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
2810	2024-12-20T23:03:32.499Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2811	2024-12-20T23:03:32.500Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b91d22f4-53a5-4d46-a4ce-4ae343529a5e", "content-length": "26", "date": "Fri, 20 Dec 2024 23:03:32 GMT"} })
2812	2024-12-20T23:03:32.500Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2813	2024-12-20T23:03:32.500Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 115.200013ms iteration = 21 status = {"physical_disks_added":0}
2814	2024-12-20T23:03:32.502Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
2815	2024-12-20T23:03:32.514Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 zone_address = fd00:1122:3344:101::2
2816	2024-12-20T23:03:32.514Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2817	2024-12-20T23:03:32.514Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d068a343-dd7d-4430-ac8e-eb956ce952d3", "content-length": "26", "date": "Fri, 20 Dec 2024 23:03:32 GMT"} })
2818	2024-12-20T23:03:32.514Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2819	2024-12-20T23:03:32.514Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
2820	2024-12-20T23:03:32.514Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
2821	2024-12-20T23:03:32.515Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "9262397a-98c5-4805-9638-a9194f0c5bbd", "content-length": "372", "date": "Fri, 20 Dec 2024 23:03:32 GMT"} })
2822	2024-12-20T23:03:32.515Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2823	2024-12-20T23:03:32.515Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2824	2024-12-20T23:03:32.516Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
2825	2024-12-20T23:03:32.517Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2826	2024-12-20T23:03:32.518Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2827	2024-12-20T23:03:32.518Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2828	2024-12-20T23:03:32.519Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
2829	2024-12-20T23:03:32.519Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2830	2024-12-20T23:03:32.519Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2831	2024-12-20T23:03:32.521Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2832	2024-12-20T23:03:32.523Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
2833	2024-12-20T23:03:32.523Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2834	2024-12-20T23:03:32.523Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2835	2024-12-20T23:03:32.531Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
2836	2024-12-20T23:03:32.531Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
2837	2024-12-20T23:03:32.532Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "35feb0c2-a8cd-400a-8a7b-8c0078e6328e", "date": "Fri, 20 Dec 2024 23:03:32 GMT"} })
2838	2024-12-20T23:03:32.532Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2839	2024-12-20T23:03:32.532Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2840	2024-12-20T23:03:32.534Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2841	2024-12-20T23:03:32.534Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2842	2024-12-20T23:03:32.538Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2843	2024-12-20T23:03:32.538Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2844	2024-12-20T23:03:32.541Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
2845	2024-12-20T23:03:32.541Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
2846	2024-12-20T23:03:32.542Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "e321549f-4f90-4cfc-8ef1-c958b4dc4d68", "date": "Fri, 20 Dec 2024 23:03:32 GMT"} })
2847	2024-12-20T23:03:32.544Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 158.738849ms iteration = 20 status = {}
2848	2024-12-20T23:03:32.548Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2849	2024-12-20T23:03:32.551Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 287d6774-d3e6-4b73-8c77-c90534d4a54d, lookup_type: ById(287d6774-d3e6-4b73-8c77-c90534d4a54d) } result = Ok(())
2850	2024-12-20T23:03:32.551Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2851	2024-12-20T23:03:32.551Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2852	2024-12-20T23:03:32.553Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 167.380545ms iteration = 20 status = 0
2853	2024-12-20T23:03:32.584Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 198.672516ms iteration = 20 status = {}
2854	2024-12-20T23:03:32.591Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
2855	2024-12-20T23:03:32.591Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "55451f29-148c-4b99-a6f1-3630ddad9ebd", "content-length": "1", "date": "Fri, 20 Dec 2024 23:03:32 GMT"} })
2856	2024-12-20T23:03:32.593Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
2857	2024-12-20T23:03:32.593Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
2858	2024-12-20T23:03:32.595Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 210.009966ms iteration = 20 status = null
2859	2024-12-20T23:03:32.624Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
2860	2024-12-20T23:03:32.625Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "c39283d8-61d7-4212-b888-15acf71d0a25", "content-length": "691", "date": "Fri, 20 Dec 2024 23:03:32 GMT"} })
2861	2024-12-20T23:03:32.625Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
2862	2024-12-20T23:03:32.625Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
2863	2024-12-20T23:03:32.625Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
2864	2024-12-20T23:03:32.626Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 240.665019ms iteration = 20 status = {}
2865	2024-12-20T23:03:32.667Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2866	2024-12-20T23:03:32.667Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2867	2024-12-20T23:03:32.669Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 03dd12a0-dc2d-46e1-a7f4-d3397b9b7aa7, rack_id: b394d0c6-19a8-4136-9148-af6c47c1f8e0, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(560ca89c-7c0c-4921-8e5c-b57676f695b0) } rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
2868	2024-12-20T23:03:32.669Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2869	2024-12-20T23:03:32.669Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2870	2024-12-20T23:03:32.678Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
2871	2024-12-20T23:03:32.679Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "43d29932-67fb-4a57-8a19-67dfb3b1ae81", "content-length": "134", "date": "Fri, 20 Dec 2024 23:03:32 GMT"} })
2872	2024-12-20T23:03:32.679Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
2873	2024-12-20T23:03:32.679Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 560ca89c-7c0c-4921-8e5c-b57676f695b0
2874	2024-12-20T23:03:32.679Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
2875	2024-12-20T23:03:32.679Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
2876	2024-12-20T23:03:32.679Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2877	2024-12-20T23:03:32.679Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2878	2024-12-20T23:03:32.679Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2879	2024-12-20T23:03:32.679Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
2880	2024-12-20T23:03:32.679Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2881	2024-12-20T23:03:32.679Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
2882	2024-12-20T23:03:32.680Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
2883	2024-12-20T23:03:32.680Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2884	2024-12-20T23:03:32.680Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2885	2024-12-20T23:03:32.682Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
2886	2024-12-20T23:03:32.682Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "18b4744c-0dab-4024-b730-de733a5d3c65", "content-length": "2", "date": "Fri, 20 Dec 2024 23:03:32 GMT"} })
2887	2024-12-20T23:03:32.682Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
2888	2024-12-20T23:03:32.682Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "dcce6824-1c6b-4104-a597-7a859b8ecc96", "content-length": "52", "date": "Fri, 20 Dec 2024 23:03:32 GMT"} })
2889	2024-12-20T23:03:32.682Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
2890	2024-12-20T23:03:32.682Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
2891	2024-12-20T23:03:32.810Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "2ec2fd05-c867-418a-b2ed-36d55749be0b", "date": "Fri, 20 Dec 2024 23:03:32 GMT"} })
2892	2024-12-20T23:03:32.810Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch = Switch0
2893	2024-12-20T23:03:32.810Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
2894	2024-12-20T23:03:32.810Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "b282ddb4-4ce8-4f0f-a5ce-abb44a17f0d3", "content-length": "682", "date": "Fri, 20 Dec 2024 23:03:32 GMT"} })
2895	2024-12-20T23:03:32.810Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2896	2024-12-20T23:03:32.810Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2897	2024-12-20T23:03:32.814Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2898	2024-12-20T23:03:32.814Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2899	2024-12-20T23:03:32.818Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2900	2024-12-20T23:03:32.818Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2901	2024-12-20T23:03:32.820Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2902	2024-12-20T23:03:32.820Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2903	2024-12-20T23:03:32.827Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 441.956866ms iteration = 20 status = {}
2904	2024-12-20T23:03:33.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 293 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 18c13b0f-bafc-49d9-96a1-58dce2ae5dc0 response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
2905	2024-12-20T23:03:42.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 59 reason = Timeout
2906	2024-12-20T23:03:42.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2907	2024-12-20T23:03:42.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 59 reason = Timeout
2908	2024-12-20T23:03:42.385Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2909	2024-12-20T23:03:42.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2910	2024-12-20T23:03:42.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2911	2024-12-20T23:03:42.386Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2912	2024-12-20T23:03:42.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2913	2024-12-20T23:03:42.389Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2914	2024-12-20T23:03:42.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2915	2024-12-20T23:03:42.395Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2916	2024-12-20T23:03:42.395Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2917	2024-12-20T23:03:42.397Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2918	2024-12-20T23:03:42.399Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
2919	2024-12-20T23:03:42.399Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2920	2024-12-20T23:03:42.399Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2921	2024-12-20T23:03:42.412Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
2922	2024-12-20T23:03:42.412Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.568047ms iteration = 59 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
2923	2024-12-20T23:03:42.422Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2924	2024-12-20T23:03:42.422Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2925	2024-12-20T23:03:42.451Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 66.589588ms iteration = 59 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
2926	2024-12-20T23:03:43.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 257 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 4d481437-def4-4326-b11f-043f0ef204dd response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
2927	2024-12-20T23:03:52.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 60 reason = Timeout
2928	2024-12-20T23:03:52.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2929	2024-12-20T23:03:52.384Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 60 reason = Timeout
2930	2024-12-20T23:03:52.384Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2931	2024-12-20T23:03:52.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2932	2024-12-20T23:03:52.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2933	2024-12-20T23:03:52.386Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2934	2024-12-20T23:03:52.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2935	2024-12-20T23:03:52.387Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2936	2024-12-20T23:03:52.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2937	2024-12-20T23:03:52.389Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2938	2024-12-20T23:03:52.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2939	2024-12-20T23:03:52.391Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2940	2024-12-20T23:03:52.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2941	2024-12-20T23:03:52.393Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2942	2024-12-20T23:03:52.395Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
2943	2024-12-20T23:03:52.395Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2944	2024-12-20T23:03:52.395Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2945	2024-12-20T23:03:52.403Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
2946	2024-12-20T23:03:52.403Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.146126ms iteration = 60 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
2947	2024-12-20T23:03:52.418Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 33.51557ms iteration = 60 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
2948	2024-12-20T23:03:53.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 295 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 4a05d25a-6505-4d2d-8554-dd549b23a99e response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
2949	2024-12-20T23:04:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 21 reason = Timeout
2950	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
2951	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 21 reason = Timeout
2952	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
2953	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2954	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 21 reason = Timeout
2955	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = saga_recovery iteration = 2 reason = Timeout
2956	2024-12-20T23:04:02.385Z	DEBG	nexus (SagaRecoverer): listing candidate sagas for recovery
2957	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
2958	2024-12-20T23:04:02.385Z	DEBG	nexus (SagaRecoverer): roles roles = RoleSet { roles: {} }
2959	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 11 reason = Timeout
2960	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 11 reason = Timeout
2961	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
2962	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 21 reason = Timeout
2963	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
2964	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 3 reason = Timeout
2965	2024-12-20T23:04:02.385Z	DEBG	nexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
2966	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 11 reason = Timeout
2967	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 11 reason = Timeout
2968	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
2969	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2970	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2971	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
2972	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 11 reason = Timeout
2973	2024-12-20T23:04:02.385Z	INFO	nexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
2974	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 11 reason = Timeout
2975	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
2976	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
2977	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2978	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 21 reason = Timeout
2979	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
2980	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 21 reason = Timeout
2981	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
2982	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 61 reason = Timeout
2983	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 12 reason = Timeout
2984	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2985	2024-12-20T23:04:02.385Z	WARN	nexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7
2986	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 13.7\u{b5}s iteration = 12 status = {"enabled":false,"target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7"}
2987	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = inventory_collection iteration = 2 reason = Timeout
2988	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 21 reason = Timeout
2989	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 22 reason = Timeout
2990	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
2991	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2992	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 11 reason = Timeout
2993	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 21 reason = Timeout
2994	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
2995	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 61 reason = Timeout
2996	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2997	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 21 reason = Timeout
2998	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2999	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 13 reason = Timeout
3000	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = external_endpoints iteration = 11 reason = Timeout
3001	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 11 reason = Timeout
3002	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3003	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
3004	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 13 reason = Timeout
3005	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 21 reason = Timeout
3006	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 11 reason = Timeout
3007	2024-12-20T23:04:02.385Z	INFO	nexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-20 22:54:02.385395808 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
3008	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3009	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 21 reason = Timeout
3010	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
3011	2024-12-20T23:04:02.385Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
3012	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3013	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 21 reason = Timeout
3014	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3015	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 21 reason = Timeout
3016	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3017	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 21 reason = Timeout
3018	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
3019	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
3020	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
3021	2024-12-20T23:04:02.385Z	DEBG	nexus (SagaRecoverer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a, .. }) resource = Database result = Ok(())
3022	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
3023	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
3024	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
3025	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3026	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3027	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3028	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
3029	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
3030	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
3031	2024-12-20T23:04:02.385Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1e296b8d-6800-4056-8034-a1ec2d005fff.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("ace0592c-7755-46f8-b47a-c9b6444fe64c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("ace0592c-7755-46f8-b47a-c9b6444fe64c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1257, tv_nsec: 775543850 } })
3032	2024-12-20T23:04:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
3033	2024-12-20T23:04:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3034	2024-12-20T23:04:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
3035	2024-12-20T23:04:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3036	2024-12-20T23:04:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
3037	2024-12-20T23:04:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3038	2024-12-20T23:04:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
3039	2024-12-20T23:04:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3040	2024-12-20T23:04:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3041	2024-12-20T23:04:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
3042	2024-12-20T23:04:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
3043	2024-12-20T23:04:02.386Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("616ce87c-1f39-49d9-ba35-ee63b2650ecd.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b4615881-2f68-4c82-be17-071bf3ae3641.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c9217e92-0af3-4930-acb2-aa82ded5b782.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("c9217e92-0af3-4930-acb2-aa82ded5b782.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 1257, tv_nsec: 775708464 } })
3044	2024-12-20T23:04:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3045	2024-12-20T23:04:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3046	2024-12-20T23:04:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3047	2024-12-20T23:04:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3048	2024-12-20T23:04:02.386Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
3049	2024-12-20T23:04:02.386Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.46397ms iteration = 11 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
3050	2024-12-20T23:04:02.386Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
3051	2024-12-20T23:04:02.386Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.411049ms iteration = 11 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
3052	2024-12-20T23:04:02.388Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1257, tv_nsec: 778168842 } })
3053	2024-12-20T23:04:02.388Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
3054	2024-12-20T23:04:02.389Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.884576ms iteration = 21 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
3055	2024-12-20T23:04:02.390Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3056	2024-12-20T23:04:02.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3057	2024-12-20T23:04:02.391Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3058	2024-12-20T23:04:02.391Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3059	2024-12-20T23:04:02.393Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3060	2024-12-20T23:04:02.393Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3061	2024-12-20T23:04:02.393Z	DEBG	nexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3062	2024-12-20T23:04:02.395Z	DEBG	nexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
3063	2024-12-20T23:04:02.395Z	DEBG	nexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
3064	2024-12-20T23:04:02.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
3065	2024-12-20T23:04:02.397Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3066	2024-12-20T23:04:02.398Z	INFO	nexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-20 22:54:02.385395808 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
3067	2024-12-20T23:04:02.398Z	DEBG	nexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 12.989157ms iteration = 11 status = {"expiration":"2024-12-20T22:54:02.385395808Z","pruned":[]}
3068	2024-12-20T23:04:02.399Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1257, tv_nsec: 788744293 } })
3069	2024-12-20T23:04:02.399Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
3070	2024-12-20T23:04:02.399Z	DEBG	nexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 14.170049ms iteration = 11 status = {"errors":[],"found_port_ok":[]}
3071	2024-12-20T23:04:02.400Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
3072	2024-12-20T23:04:02.400Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3073	2024-12-20T23:04:02.400Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3074	2024-12-20T23:04:02.403Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3075	2024-12-20T23:04:02.408Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
3076	2024-12-20T23:04:02.408Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3077	2024-12-20T23:04:02.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3078	2024-12-20T23:04:02.412Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3079	2024-12-20T23:04:02.415Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3080	2024-12-20T23:04:02.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
3081	2024-12-20T23:04:02.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3082	2024-12-20T23:04:02.418Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3083	2024-12-20T23:04:02.418Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3084	2024-12-20T23:04:02.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3085	2024-12-20T23:04:02.422Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
3086	2024-12-20T23:04:02.425Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3087	2024-12-20T23:04:02.425Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3088	2024-12-20T23:04:02.427Z	DEBG	nexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(82b347be-63bb-4e68-a378-fce0897b089f, 0), (d4331d00-3259-4f38-a364-759e1a8174d3, 0), (6d2fdc84-2f6a-4beb-9e43-4e946f1fc173, 0)]
3089	2024-12-20T23:04:02.428Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3090	2024-12-20T23:04:02.428Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1257, tv_nsec: 817755619 } })
3091	2024-12-20T23:04:02.428Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3092	2024-12-20T23:04:02.429Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3093	2024-12-20T23:04:02.430Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 45.009696ms iteration = 61 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
3094	2024-12-20T23:04:02.432Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
3095	2024-12-20T23:04:02.433Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3096	2024-12-20T23:04:02.433Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3097	2024-12-20T23:04:02.434Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3098	2024-12-20T23:04:02.434Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3099	2024-12-20T23:04:02.435Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3100	2024-12-20T23:04:02.436Z	DEBG	nexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
3101	2024-12-20T23:04:02.436Z	INFO	nexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
3102	2024-12-20T23:04:02.436Z	DEBG	nexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 51.054642ms iteration = 11 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
3103	2024-12-20T23:04:02.436Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
3104	2024-12-20T23:04:02.436Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
3105	2024-12-20T23:04:02.436Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
3106	2024-12-20T23:04:02.436Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
3107	2024-12-20T23:04:02.437Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
3108	2024-12-20T23:04:02.437Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
3109	2024-12-20T23:04:02.438Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
3110	2024-12-20T23:04:02.438Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3111	2024-12-20T23:04:02.439Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3112	2024-12-20T23:04:02.440Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3113	2024-12-20T23:04:02.441Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3114	2024-12-20T23:04:02.442Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
3115	2024-12-20T23:04:02.442Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
3116	2024-12-20T23:04:02.442Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
3117	2024-12-20T23:04:02.442Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
3118	2024-12-20T23:04:02.443Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
3119	2024-12-20T23:04:02.443Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
3120	2024-12-20T23:04:02.443Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3121	2024-12-20T23:04:02.448Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 63.277758ms iteration = 21 status = {"errors":[],"finish_invoked_ok":[]}
3122	2024-12-20T23:04:02.448Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
3123	2024-12-20T23:04:02.448Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3124	2024-12-20T23:04:02.448Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3125	2024-12-20T23:04:02.450Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 65.013315ms iteration = 21 status = {"errors":[],"garbage_collect_requested":[]}
3126	2024-12-20T23:04:02.455Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3127	2024-12-20T23:04:02.455Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3128	2024-12-20T23:04:02.455Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3129	2024-12-20T23:04:02.455Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3130	2024-12-20T23:04:02.458Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
3131	2024-12-20T23:04:02.458Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3132	2024-12-20T23:04:02.458Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3133	2024-12-20T23:04:02.458Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
3134	2024-12-20T23:04:02.458Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3135	2024-12-20T23:04:02.458Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3136	2024-12-20T23:04:02.459Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3137	2024-12-20T23:04:02.460Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3138	2024-12-20T23:04:02.461Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3139	2024-12-20T23:04:02.462Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3140	2024-12-20T23:04:02.462Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
3141	2024-12-20T23:04:02.462Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
3142	2024-12-20T23:04:02.462Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3143	2024-12-20T23:04:02.462Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1257, tv_nsec: 851996769 } })
3144	2024-12-20T23:04:02.462Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
3145	2024-12-20T23:04:02.462Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
3146	2024-12-20T23:04:02.462Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
3147	2024-12-20T23:04:02.462Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
3148	2024-12-20T23:04:02.462Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
3149	2024-12-20T23:04:02.463Z	INFO	nexus (SagaRecoverer): listed in-progress sagas count = 0 file = nexus/src/app/background/tasks/saga_recovery.rs:438
3150	2024-12-20T23:04:02.463Z	DEBG	nexus (BackgroundTasks): activation complete background_task = saga_recovery elapsed = 78.119455ms iteration = 2 status = {"last_pass":{"Success":{"nfailed":0,"nfound":0,"nrecovered":0,"nremoved":0,"nskipped":0}},"ntotal_failures":0,"ntotal_finished":0,"ntotal_recovered":0,"ntotal_sec_errors_bad_state":0,"ntotal_sec_errors_missing":0,"ntotal_started":0,"recent_failures":[],"recent_recoveries":[]}
3151	2024-12-20T23:04:02.464Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
3152	2024-12-20T23:04:02.464Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3153	2024-12-20T23:04:02.465Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3154	2024-12-20T23:04:02.466Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3155	2024-12-20T23:04:02.466Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
3156	2024-12-20T23:04:02.467Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3157	2024-12-20T23:04:02.467Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
3158	2024-12-20T23:04:02.467Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3159	2024-12-20T23:04:02.467Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3160	2024-12-20T23:04:02.467Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
3161	2024-12-20T23:04:02.467Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
3162	2024-12-20T23:04:02.467Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
3163	2024-12-20T23:04:02.467Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3164	2024-12-20T23:04:02.468Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3165	2024-12-20T23:04:02.468Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
3166	2024-12-20T23:04:02.468Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3167	2024-12-20T23:04:02.468Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3168	2024-12-20T23:04:02.471Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3169	2024-12-20T23:04:02.471Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3170	2024-12-20T23:04:02.471Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3171	2024-12-20T23:04:02.471Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
3172	2024-12-20T23:04:02.471Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
3173	2024-12-20T23:04:02.471Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
3174	2024-12-20T23:04:02.471Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3175	2024-12-20T23:04:02.472Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3176	2024-12-20T23:04:02.473Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3177	2024-12-20T23:04:02.474Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3178	2024-12-20T23:04:02.474Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3179	2024-12-20T23:04:02.475Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3180	2024-12-20T23:04:02.476Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 90.862304ms iteration = 22 status = {"physical_disks_added":0}
3181	2024-12-20T23:04:02.477Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
3182	2024-12-20T23:04:02.477Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3183	2024-12-20T23:04:02.480Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3184	2024-12-20T23:04:02.480Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
3185	2024-12-20T23:04:02.480Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3186	2024-12-20T23:04:02.481Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3187	2024-12-20T23:04:02.482Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3188	2024-12-20T23:04:02.485Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
3189	2024-12-20T23:04:02.485Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3190	2024-12-20T23:04:02.485Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3191	2024-12-20T23:04:02.486Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
3192	2024-12-20T23:04:02.486Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
3193	2024-12-20T23:04:02.487Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
3194	2024-12-20T23:04:02.487Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 1e86bf23-6ffd-4a12-93c7-0061d7d17590 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
3195	2024-12-20T23:04:02.487Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3196	2024-12-20T23:04:02.487Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3197	2024-12-20T23:04:02.488Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 103.747659ms iteration = 21 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
3198	2024-12-20T23:04:02.489Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3199	2024-12-20T23:04:02.492Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3200	2024-12-20T23:04:02.492Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3201	2024-12-20T23:04:02.492Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
3202	2024-12-20T23:04:02.492Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3203	2024-12-20T23:04:02.493Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3204	2024-12-20T23:04:02.493Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
3205	2024-12-20T23:04:02.493Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
3206	2024-12-20T23:04:02.493Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 108.673584ms iteration = 21 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
3207	2024-12-20T23:04:02.493Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3208	2024-12-20T23:04:02.495Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 1e86bf23-6ffd-4a12-93c7-0061d7d17590 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
3209	2024-12-20T23:04:02.495Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
3210	2024-12-20T23:04:02.495Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-20 22:54:38.956071 UTC dns_group = internal generation = 1
3211	2024-12-20T23:04:02.495Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 110.152374ms iteration = 11 status = {"generation":1}
3212	2024-12-20T23:04:02.495Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
3213	2024-12-20T23:04:02.496Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3214	2024-12-20T23:04:02.496Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3215	2024-12-20T23:04:02.499Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
3216	2024-12-20T23:04:02.499Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
3217	2024-12-20T23:04:02.500Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3218	2024-12-20T23:04:02.501Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3219	2024-12-20T23:04:02.501Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 116.083837ms iteration = 21 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
3220	2024-12-20T23:04:02.501Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
3221	2024-12-20T23:04:02.501Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 116.334355ms iteration = 61 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
3222	2024-12-20T23:04:02.503Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3223	2024-12-20T23:04:02.503Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3224	2024-12-20T23:04:02.504Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3225	2024-12-20T23:04:02.504Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3226	2024-12-20T23:04:02.505Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3227	2024-12-20T23:04:02.505Z	DEBG	nexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
3228	2024-12-20T23:04:02.505Z	INFO	nexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
3229	2024-12-20T23:04:02.505Z	DEBG	nexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 120.316374ms iteration = 11 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
3230	2024-12-20T23:04:02.507Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 287d6774-d3e6-4b73-8c77-c90534d4a54d, lookup_type: ById(287d6774-d3e6-4b73-8c77-c90534d4a54d) } result = Ok(())
3231	2024-12-20T23:04:02.507Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3232	2024-12-20T23:04:02.508Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3233	2024-12-20T23:04:02.510Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 124.621402ms iteration = 21 status = 0
3234	2024-12-20T23:04:02.518Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
3235	2024-12-20T23:04:02.518Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 5fa20c62-75c0-4531-be07-51b87d303d18 dns_zone_name = oxide.test found_so_far = 0 version = 2
3236	2024-12-20T23:04:02.518Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3237	2024-12-20T23:04:02.519Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3238	2024-12-20T23:04:02.520Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3239	2024-12-20T23:04:02.522Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 137.616639ms iteration = 21 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
3240	2024-12-20T23:04:02.524Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
3241	2024-12-20T23:04:02.524Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3242	2024-12-20T23:04:02.524Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3243	2024-12-20T23:04:02.524Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
3244	2024-12-20T23:04:02.524Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 139.506351ms iteration = 21 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
3245	2024-12-20T23:04:02.527Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 5fa20c62-75c0-4531-be07-51b87d303d18 dns_zone_name = oxide.test found_so_far = 1 version = 2
3246	2024-12-20T23:04:02.527Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
3247	2024-12-20T23:04:02.527Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-20 22:54:39.899407 UTC dns_group = external generation = 2
3248	2024-12-20T23:04:02.527Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 141.957558ms iteration = 11 status = {"generation":2}
3249	2024-12-20T23:04:02.531Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
3250	2024-12-20T23:04:02.543Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
3251	2024-12-20T23:04:02.543Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3252	2024-12-20T23:04:02.543Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "11e3adc0-bbe5-4366-afe1-358b31e31f8b", "content-length": "26", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} })
3253	2024-12-20T23:04:02.543Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3254	2024-12-20T23:04:02.544Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3255	2024-12-20T23:04:02.545Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3256	2024-12-20T23:04:02.546Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3257	2024-12-20T23:04:02.553Z	DEBG	nexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-20T22:53:58.073373Z, time_modified: 2024-12-20T22:53:58.073373Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, db_silo: Silo { identity: SiloIdentity { id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-20T22:54:39.847015Z, time_modified: 2024-12-20T22:54:39.847015Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, db_silo: Silo { identity: SiloIdentity { id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-20T22:54:39.847015Z, time_modified: 2024-12-20T22:54:39.847015Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156" }] }) }
3258	2024-12-20T23:04:02.553Z	DEBG	nexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 168.359563ms iteration = 11 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"ee579470-b16b-4af1-ad3a-b3d29f9a2fb2","tls_certs":["ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156"]}},"default_endpoint":{"silo_id":"ee579470-b16b-4af1-ad3a-b3d29f9a2fb2","tls_certs":["ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
3259	2024-12-20T23:04:02.553Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
3260	2024-12-20T23:04:02.556Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3261	2024-12-20T23:04:02.560Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
3262	2024-12-20T23:04:02.560Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3263	2024-12-20T23:04:02.560Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3264	2024-12-20T23:04:02.571Z	WARN	nexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
3265	2024-12-20T23:04:02.571Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 186.305366ms iteration = 11 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
3266	2024-12-20T23:04:02.582Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
3267	2024-12-20T23:04:02.582Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3268	2024-12-20T23:04:02.582Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "145ee5d1-4dec-4821-8844-ee6a5c23d5c4", "content-length": "26", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} })
3269	2024-12-20T23:04:02.582Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3270	2024-12-20T23:04:02.584Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3271	2024-12-20T23:04:02.649Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 zone_address = fd00:1122:3344:101::2
3272	2024-12-20T23:04:02.649Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3273	2024-12-20T23:04:02.649Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1370107c-1d23-4eac-bcdf-839d5cfb8322", "content-length": "26", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} })
3274	2024-12-20T23:04:02.649Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3275	2024-12-20T23:04:02.651Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
3276	2024-12-20T23:04:02.663Z	DEBG	nexus (BackgroundTasks): begin collection background_task = inventory_collection
3277	2024-12-20T23:04:02.663Z	DEBG	nexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
3278	2024-12-20T23:04:02.663Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
3279	2024-12-20T23:04:02.664Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "855eccdf-88ef-404e-a53c-24e6481995b6", "content-length": "476", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} })
3280	2024-12-20T23:04:02.664Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
3281	2024-12-20T23:04:02.664Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "062567e9-37df-4fea-b273-efa994112bb6", "content-length": "734", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} })
3282	2024-12-20T23:04:02.664Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
3283	2024-12-20T23:04:02.664Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "df25d6ba-1a8e-4492-ac8a-311791431352", "content-length": "111", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} })
3284	2024-12-20T23:04:02.664Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
3285	2024-12-20T23:04:02.665Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d35e7e68-9d0b-40f8-a0f3-644f22742a06", "content-length": "111", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} })
3286	2024-12-20T23:04:02.665Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
3287	2024-12-20T23:04:02.665Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "78ec45dc-bff5-42b4-b764-310c7040169d", "content-length": "167", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} })
3288	2024-12-20T23:04:02.665Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
3289	2024-12-20T23:04:02.665Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0e8d71c-7bf3-4752-9cd5-10566aa3e626", "content-length": "167", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} })
3290	2024-12-20T23:04:02.665Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
3291	2024-12-20T23:04:02.665Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e1a6f246-6e74-43ce-a507-b5553ad7aeae", "content-length": "175", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} })
3292	2024-12-20T23:04:02.665Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
3293	2024-12-20T23:04:02.666Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0ee59e7-0652-4b92-97dc-d06a8d68a15c", "content-length": "175", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} })
3294	2024-12-20T23:04:02.666Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
3295	2024-12-20T23:04:02.666Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd63593c-aa3b-4225-8692-37ef7b130d5f", "content-length": "702", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} })
3296	2024-12-20T23:04:02.666Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
3297	2024-12-20T23:04:02.666Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d4ef0ab-52b2-402d-9d68-7fbd71af56df", "content-length": "727", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} })
3298	2024-12-20T23:04:02.666Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
3299	2024-12-20T23:04:02.666Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4b53b16-e831-430d-83bd-fb4df62ab231", "content-length": "729", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} })
3300	2024-12-20T23:04:02.666Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
3301	2024-12-20T23:04:02.666Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "766a69e7-fac0-4b79-863d-d708fb014938", "content-length": "728", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} })
3302	2024-12-20T23:04:02.666Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
3303	2024-12-20T23:04:02.667Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5a79621a-9f8c-4375-9aa7-3fe67ab00a2c", "content-length": "722", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} })
3304	2024-12-20T23:04:02.667Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
3305	2024-12-20T23:04:02.667Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "99017e0f-f4cb-48fe-89dd-49ed1714e881", "content-length": "109", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} })
3306	2024-12-20T23:04:02.667Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
3307	2024-12-20T23:04:02.667Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b0906b5-0cfc-4f69-a85e-24a0553d7f36", "content-length": "109", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} })
3308	2024-12-20T23:04:02.667Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
3309	2024-12-20T23:04:02.668Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "aff2f37f-7ade-4f04-8fbd-aaaf5e552453", "content-length": "169", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} })
3310	2024-12-20T23:04:02.668Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
3311	2024-12-20T23:04:02.668Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a0a16199-cbdd-4081-9766-5887f5b631c4", "content-length": "169", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} })
3312	2024-12-20T23:04:02.668Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
3313	2024-12-20T23:04:02.668Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ff29a80-06a1-438b-a32a-a47a0085f1da", "content-length": "178", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} })
3314	2024-12-20T23:04:02.668Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
3315	2024-12-20T23:04:02.668Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f1593970-9c1b-4ce7-9ad9-c9b09709551c", "content-length": "178", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} })
3316	2024-12-20T23:04:02.668Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
3317	2024-12-20T23:04:02.669Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "303a780b-e9c6-4bf3-9e5b-d6e88a2a6b8f", "content-length": "702", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} })
3318	2024-12-20T23:04:02.669Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
3319	2024-12-20T23:04:02.669Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "425eb6aa-34b3-467d-9583-0703479bedd5", "content-length": "727", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} })
3320	2024-12-20T23:04:02.669Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
3321	2024-12-20T23:04:02.669Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3488b6e7-fc67-4ac8-864a-2158aca9afb2", "content-length": "729", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} })
3322	2024-12-20T23:04:02.669Z	DEBG	nexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
3323	2024-12-20T23:04:02.669Z	DEBG	nexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b491b96e-e934-4d58-af9d-887bd0eed1a3", "content-length": "728", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} })
3324	2024-12-20T23:04:02.669Z	DEBG	nexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
3325	2024-12-20T23:04:02.670Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
3326	2024-12-20T23:04:02.714Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
3327	2024-12-20T23:04:02.715Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "28e12f5f-3fa5-4947-aadf-c4b70c2f7202", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
3328	2024-12-20T23:04:02.715Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
3329	2024-12-20T23:04:02.717Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "abacceb0-47e0-4d89-a760-8aba6fb3483b", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
3330	2024-12-20T23:04:02.717Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
3331	2024-12-20T23:04:02.718Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
3332	2024-12-20T23:04:02.718Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 333.514678ms iteration = 13 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
3333	2024-12-20T23:04:02.731Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
3334	2024-12-20T23:04:02.731Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
3335	2024-12-20T23:04:02.731Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e3c2b51-c7da-44a3-8db7-79179b7ef26b", "content-length": "372", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} })
3336	2024-12-20T23:04:02.731Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3337	2024-12-20T23:04:02.731Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
3338	2024-12-20T23:04:02.731Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3339	2024-12-20T23:04:02.733Z	DEBG	nexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3340	2024-12-20T23:04:02.734Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3341	2024-12-20T23:04:02.734Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3342	2024-12-20T23:04:02.735Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
3343	2024-12-20T23:04:02.735Z	DEBG	nexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
3344	2024-12-20T23:04:02.735Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
3345	2024-12-20T23:04:02.736Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3346	2024-12-20T23:04:02.736Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3347	2024-12-20T23:04:02.737Z	DEBG	nexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3348	2024-12-20T23:04:02.738Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3349	2024-12-20T23:04:02.739Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
3350	2024-12-20T23:04:02.739Z	DEBG	nexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
3351	2024-12-20T23:04:02.739Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3352	2024-12-20T23:04:02.739Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
3353	2024-12-20T23:04:02.741Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
3354	2024-12-20T23:04:02.742Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "54a9c5a0-968b-487a-a20c-4b2364e2ae5a", "content-length": "1", "date": "Fri, 20 Dec 2024 23:04:01 GMT"} })
3355	2024-12-20T23:04:02.749Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3356	2024-12-20T23:04:02.749Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3357	2024-12-20T23:04:02.750Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3358	2024-12-20T23:04:02.750Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3359	2024-12-20T23:04:02.752Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 367.077989ms iteration = 21 status = null
3360	2024-12-20T23:04:02.753Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3361	2024-12-20T23:04:02.753Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3362	2024-12-20T23:04:02.756Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3363	2024-12-20T23:04:02.758Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
3364	2024-12-20T23:04:02.758Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3365	2024-12-20T23:04:02.758Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3366	2024-12-20T23:04:02.764Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 379.800289ms iteration = 21 status = {}
3367	2024-12-20T23:04:02.800Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
3368	2024-12-20T23:04:02.800Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
3369	2024-12-20T23:04:02.801Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "77e2ffcb-c6de-4c87-84ee-dd950adf31da", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} })
3370	2024-12-20T23:04:02.801Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3371	2024-12-20T23:04:02.802Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3372	2024-12-20T23:04:02.804Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3373	2024-12-20T23:04:02.804Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3374	2024-12-20T23:04:02.833Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
3375	2024-12-20T23:04:02.834Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "a3703207-1ae1-49c6-8f16-d836224d16bb", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
3376	2024-12-20T23:04:02.834Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
3377	2024-12-20T23:04:02.836Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "44574254-10be-4c46-b043-d6f4c4fa5a7b", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
3378	2024-12-20T23:04:02.836Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
3379	2024-12-20T23:04:02.838Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "24f93eaa-1d83-46d1-b1cd-3f86a39316cb", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
3380	2024-12-20T23:04:02.838Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
3381	2024-12-20T23:04:02.840Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
3382	2024-12-20T23:04:02.840Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 454.940732ms iteration = 13 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
3383	2024-12-20T23:04:02.858Z	DEBG	nexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
3384	2024-12-20T23:04:02.858Z	DEBG	nexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
3385	2024-12-20T23:04:02.860Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3386	2024-12-20T23:04:02.860Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3387	2024-12-20T23:04:02.861Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3388	2024-12-20T23:04:02.861Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3389	2024-12-20T23:04:02.862Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3390	2024-12-20T23:04:02.864Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3391	2024-12-20T23:04:02.865Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3392	2024-12-20T23:04:02.866Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
3393	2024-12-20T23:04:02.866Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3394	2024-12-20T23:04:02.867Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3395	2024-12-20T23:04:02.868Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
3396	2024-12-20T23:04:02.868Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
3397	2024-12-20T23:04:02.869Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3398	2024-12-20T23:04:02.869Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3399	2024-12-20T23:04:02.870Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3400	2024-12-20T23:04:02.871Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3401	2024-12-20T23:04:02.872Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "b924e2f4-5c2a-4f06-ba03-59b3158a62ef", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} })
3402	2024-12-20T23:04:02.873Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3403	2024-12-20T23:04:02.873Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3404	2024-12-20T23:04:02.874Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 489.222103ms iteration = 21 status = {}
3405	2024-12-20T23:04:02.875Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3406	2024-12-20T23:04:02.876Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3407	2024-12-20T23:04:02.876Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3408	2024-12-20T23:04:02.877Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3409	2024-12-20T23:04:02.878Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3410	2024-12-20T23:04:02.878Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3411	2024-12-20T23:04:02.880Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3412	2024-12-20T23:04:02.883Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
3413	2024-12-20T23:04:02.883Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3414	2024-12-20T23:04:02.883Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3415	2024-12-20T23:04:02.885Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3416	2024-12-20T23:04:02.885Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3417	2024-12-20T23:04:02.886Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3418	2024-12-20T23:04:02.886Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3419	2024-12-20T23:04:02.888Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3420	2024-12-20T23:04:02.891Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
3421	2024-12-20T23:04:02.891Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3422	2024-12-20T23:04:02.891Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3423	2024-12-20T23:04:02.895Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3424	2024-12-20T23:04:02.896Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3425	2024-12-20T23:04:02.897Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3426	2024-12-20T23:04:02.898Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3427	2024-12-20T23:04:02.901Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 03dd12a0-dc2d-46e1-a7f4-d3397b9b7aa7, rack_id: b394d0c6-19a8-4136-9148-af6c47c1f8e0, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(560ca89c-7c0c-4921-8e5c-b57676f695b0) } rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
3428	2024-12-20T23:04:02.901Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3429	2024-12-20T23:04:02.901Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3430	2024-12-20T23:04:02.901Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3431	2024-12-20T23:04:02.901Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3432	2024-12-20T23:04:02.903Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3433	2024-12-20T23:04:02.903Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3434	2024-12-20T23:04:02.905Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3435	2024-12-20T23:04:02.906Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3436	2024-12-20T23:04:02.907Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3437	2024-12-20T23:04:02.908Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3438	2024-12-20T23:04:02.908Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3439	2024-12-20T23:04:02.908Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
3440	2024-12-20T23:04:02.908Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3441	2024-12-20T23:04:02.908Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3442	2024-12-20T23:04:02.910Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3443	2024-12-20T23:04:02.910Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3444	2024-12-20T23:04:02.911Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3445	2024-12-20T23:04:02.911Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3446	2024-12-20T23:04:02.912Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
3447	2024-12-20T23:04:02.912Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "295e4773-5ecd-4aed-a2ad-ef9be35f430f", "content-length": "134", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} })
3448	2024-12-20T23:04:02.913Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
3449	2024-12-20T23:04:02.913Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 560ca89c-7c0c-4921-8e5c-b57676f695b0
3450	2024-12-20T23:04:02.913Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3451	2024-12-20T23:04:02.913Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
3452	2024-12-20T23:04:02.913Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
3453	2024-12-20T23:04:02.913Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3454	2024-12-20T23:04:02.913Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3455	2024-12-20T23:04:02.913Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3456	2024-12-20T23:04:02.913Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
3457	2024-12-20T23:04:02.913Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3458	2024-12-20T23:04:02.913Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
3459	2024-12-20T23:04:02.913Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
3460	2024-12-20T23:04:02.913Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3461	2024-12-20T23:04:02.914Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3462	2024-12-20T23:04:02.916Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
3463	2024-12-20T23:04:02.916Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "6558e2fe-e1a6-4be2-a03d-ad9a5ad90b66", "content-length": "2", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} })
3464	2024-12-20T23:04:02.916Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
3465	2024-12-20T23:04:02.916Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "d8aa5172-4bf3-44e7-b96b-97e780888da0", "content-length": "52", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} })
3466	2024-12-20T23:04:02.916Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
3467	2024-12-20T23:04:02.916Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
3468	2024-12-20T23:04:02.917Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
3469	2024-12-20T23:04:02.917Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3470	2024-12-20T23:04:02.917Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3471	2024-12-20T23:04:02.918Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3472	2024-12-20T23:04:02.918Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3473	2024-12-20T23:04:02.920Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3474	2024-12-20T23:04:02.920Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3475	2024-12-20T23:04:02.921Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3476	2024-12-20T23:04:02.924Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
3477	2024-12-20T23:04:02.924Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3478	2024-12-20T23:04:02.924Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3479	2024-12-20T23:04:02.931Z	DEBG	nexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 1e21a6ac-6d39-461b-a43a-59a0e5d45557, kind: Service { id: a67567ae-7026-453e-acd2-e1780994c729 }, name: Name("nexus-a67567ae-7026-453e-acd2-e1780994c729"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 188, 28])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 2351e16d-ccc3-42df-b594-9d3a6ec23973, kind: Service { id: f3b60bd3-6e34-4140-a755-57f26dbc5b34 }, name: Name("nexus-f3b60bd3-6e34-4140-a755-57f26dbc5b34"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 142, 57])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8be612fb-34b6-43da-9363-17ad360f283a, kind: Service { id: 371fea12-332e-4334-902e-d2154e69e006 }, name: Name("nexus-371fea12-332e-4334-902e-d2154e69e006"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 100])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 9b5312da-a277-4244-baaa-33815f0464e4, kind: Service { id: ace0592c-7755-46f8-b47a-c9b6444fe64c }, name: Name("external-dns-ace0592c-7755-46f8-b47a-c9b6444fe64c"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 142, 244])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b6a4ac58-71fd-4271-af9e-04ea5fb7455a, kind: Service { id: 1e296b8d-6800-4056-8034-a1ec2d005fff }, name: Name("external-dns-1e296b8d-6800-4056-8034-a1ec2d005fff"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 157, 244])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
3480	2024-12-20T23:04:02.931Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3481	2024-12-20T23:04:02.931Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3482	2024-12-20T23:04:02.933Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3483	2024-12-20T23:04:02.934Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
3484	2024-12-20T23:04:02.934Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3485	2024-12-20T23:04:02.935Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3486	2024-12-20T23:04:02.936Z	DEBG	nexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
3487	2024-12-20T23:04:02.936Z	DEBG	nexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 9b5312da-a277-4244-baaa-33815f0464e4, kind: Service { id: ace0592c-7755-46f8-b47a-c9b6444fe64c }, name: Name("external-dns-ace0592c-7755-46f8-b47a-c9b6444fe64c"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 142, 244])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b6a4ac58-71fd-4271-af9e-04ea5fb7455a, kind: Service { id: 1e296b8d-6800-4056-8034-a1ec2d005fff }, name: Name("external-dns-1e296b8d-6800-4056-8034-a1ec2d005fff"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 157, 244])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 1e21a6ac-6d39-461b-a43a-59a0e5d45557, kind: Service { id: a67567ae-7026-453e-acd2-e1780994c729 }, name: Name("nexus-a67567ae-7026-453e-acd2-e1780994c729"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 188, 28])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 2351e16d-ccc3-42df-b594-9d3a6ec23973, kind: Service { id: f3b60bd3-6e34-4140-a755-57f26dbc5b34 }, name: Name("nexus-f3b60bd3-6e34-4140-a755-57f26dbc5b34"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 142, 57])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8be612fb-34b6-43da-9363-17ad360f283a, kind: Service { id: 371fea12-332e-4334-902e-d2154e69e006 }, name: Name("nexus-371fea12-332e-4334-902e-d2154e69e006"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 100])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
3488	2024-12-20T23:04:02.936Z	DEBG	nexus (service-firewall-rule-progator): resolved 2 rules for sleds background_task = service_firewall_rule_propagation
3489	2024-12-20T23:04:02.994Z	DEBG	nexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: 287d6774-d3e6-4b73-8c77-c90534d4a54d, time_created: 2024-12-20T22:54:34.405180Z, time_modified: 2024-12-20T22:54:34.405180Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: b394d0c6-19a8-4136-9148-af6c47c1f8e0, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(51474595840)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)) }]
3490	2024-12-20T23:04:02.994Z	DEBG	nexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
3491	2024-12-20T23:04:02.994Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3492	2024-12-20T23:04:02.995Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3493	2024-12-20T23:04:02.996Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3494	2024-12-20T23:04:02.997Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3495	2024-12-20T23:04:02.999Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3496	2024-12-20T23:04:03.001Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: 287d6774-d3e6-4b73-8c77-c90534d4a54d, lookup_type: ById(287d6774-d3e6-4b73-8c77-c90534d4a54d) } result = Ok(())
3497	2024-12-20T23:04:03.021Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
3498	2024-12-20T23:04:03.022Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "5606fb34-a559-49fc-ae7b-bb59bc6143ee", "content-length": "691", "date": "Fri, 20 Dec 2024 23:04:03 GMT"} })
3499	2024-12-20T23:04:03.022Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
3500	2024-12-20T23:04:03.022Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
3501	2024-12-20T23:04:03.022Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
3502	2024-12-20T23:04:03.024Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 638.674477ms iteration = 21 status = {}
3503	2024-12-20T23:04:03.042Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "59263832-8af6-4e20-b0f3-333aea86c505", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} })
3504	2024-12-20T23:04:03.042Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch = Switch0
3505	2024-12-20T23:04:03.042Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
3506	2024-12-20T23:04:03.042Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "77710bff-cd49-4f8d-8c23-99ccd840bcbe", "content-length": "682", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} })
3507	2024-12-20T23:04:03.042Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3508	2024-12-20T23:04:03.042Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3509	2024-12-20T23:04:03.043Z	DEBG	nexus (service-firewall-rule-progator): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
3510	2024-12-20T23:04:03.043Z	DEBG	nexus (service-firewall-rule-progator): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "1086256c-c873-4192-baf4-23086fe1e469", "date": "Fri, 20 Dec 2024 23:04:03 GMT"} })
3511	2024-12-20T23:04:03.046Z	INFO	nexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:496
3512	2024-12-20T23:04:03.046Z	DEBG	nexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 661.38172ms
3513	2024-12-20T23:04:03.046Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 661.39636ms iteration = 3 status = {}
3514	2024-12-20T23:04:03.046Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3515	2024-12-20T23:04:03.047Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3516	2024-12-20T23:04:03.050Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3517	2024-12-20T23:04:03.051Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3518	2024-12-20T23:04:03.053Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3519	2024-12-20T23:04:03.053Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3520	2024-12-20T23:04:03.060Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 675.112597ms iteration = 21 status = {}
3521	2024-12-20T23:04:03.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 285 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = f713b1eb-5c64-4865-9652-09d409ae7a9d response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
3522	2024-12-20T23:04:03.161Z	DEBG	nexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "197612f6-92f1-4399-beec-898f9298c97c", "content-length": "30798", "date": "Fri, 20 Dec 2024 23:04:02 GMT"} })
3523	2024-12-20T23:04:03.180Z	DEBG	nexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
3524	2024-12-20T23:04:03.180Z	DEBG	nexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
3525	2024-12-20T23:04:03.180Z	DEBG	nexus (BackgroundTasks): finished collection background_task = inventory_collection
3526	2024-12-20T23:04:03.185Z	DEBG	nexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
3527	2024-12-20T23:04:03.185Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
3528	2024-12-20T23:04:03.186Z	DEBG	nexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3529	2024-12-20T23:04:03.188Z	DEBG	nexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
3530	2024-12-20T23:04:03.188Z	DEBG	nexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
3531	2024-12-20T23:04:03.189Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
3532	2024-12-20T23:04:03.391Z	INFO	nexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 3c907d5d-bfe8-4021-bf59-d45ebf23c180 file = nexus/db-queries/src/db/datastore/inventory.rs:1010
3533	2024-12-20T23:04:03.391Z	DEBG	nexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 3c907d5d-bfe8-4021-bf59-d45ebf23c180 time_started = 2024-12-20 23:04:02.663695 UTC
3534	2024-12-20T23:04:03.391Z	DEBG	nexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 1.006454595s iteration = 2 status = {"collection_id":"3c907d5d-bfe8-4021-bf59-d45ebf23c180","time_done":"2024-12-20 23:04:03.182870 UTC","time_started":"2024-12-20 23:04:02.663695 UTC"}
3535	2024-12-20T23:04:03.391Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 23 reason = Dependency
3536	2024-12-20T23:04:03.391Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3537	2024-12-20T23:04:03.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3538	2024-12-20T23:04:03.393Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3539	2024-12-20T23:04:03.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
3540	2024-12-20T23:04:03.396Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3541	2024-12-20T23:04:03.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3542	2024-12-20T23:04:03.398Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3543	2024-12-20T23:04:03.399Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3544	2024-12-20T23:04:03.400Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3545	2024-12-20T23:04:03.402Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
3546	2024-12-20T23:04:03.402Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3547	2024-12-20T23:04:03.403Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3548	2024-12-20T23:04:03.446Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 54.704603ms iteration = 23 status = {"physical_disks_added":0}
3549	2024-12-20T23:04:12.384Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 62 reason = Timeout
3550	2024-12-20T23:04:12.384Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3551	2024-12-20T23:04:12.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 62 reason = Timeout
3552	2024-12-20T23:04:12.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3553	2024-12-20T23:04:12.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3554	2024-12-20T23:04:12.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3555	2024-12-20T23:04:12.387Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3556	2024-12-20T23:04:12.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3557	2024-12-20T23:04:12.389Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3558	2024-12-20T23:04:12.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3559	2024-12-20T23:04:12.391Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3560	2024-12-20T23:04:12.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3561	2024-12-20T23:04:12.393Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3562	2024-12-20T23:04:12.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
3563	2024-12-20T23:04:12.396Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3564	2024-12-20T23:04:12.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3565	2024-12-20T23:04:12.407Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
3566	2024-12-20T23:04:12.407Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.025042ms iteration = 62 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
3567	2024-12-20T23:04:12.414Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3568	2024-12-20T23:04:12.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3569	2024-12-20T23:04:12.415Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.117394ms iteration = 62 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
3570	2024-12-20T23:04:13.149Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 287 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 9bf19dc0-e3b1-451a-8745-6b118e6c2af4 response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
3571	2024-12-20T23:04:22.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 63 reason = Timeout
3572	2024-12-20T23:04:22.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3573	2024-12-20T23:04:22.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 63 reason = Timeout
3574	2024-12-20T23:04:22.385Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3575	2024-12-20T23:04:22.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3576	2024-12-20T23:04:22.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3577	2024-12-20T23:04:22.387Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3578	2024-12-20T23:04:22.387Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3579	2024-12-20T23:04:22.388Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3580	2024-12-20T23:04:22.389Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.545577ms iteration = 63 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
3581	2024-12-20T23:04:22.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3582	2024-12-20T23:04:22.390Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3583	2024-12-20T23:04:22.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3584	2024-12-20T23:04:22.392Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3585	2024-12-20T23:04:22.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3586	2024-12-20T23:04:22.394Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3587	2024-12-20T23:04:22.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
3588	2024-12-20T23:04:22.396Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3589	2024-12-20T23:04:22.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3590	2024-12-20T23:04:22.404Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
3591	2024-12-20T23:04:22.404Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.008352ms iteration = 63 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
3592	2024-12-20T23:04:23.149Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 365 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 0ff490af-c7fc-464f-b6cd-0923b1756d8d response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
3593	2024-12-20T23:04:28.138Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42603 req_id = 7f50231d-0f6a-474d-b276-eda8023b6cc9 roles = RoleSet { roles: {} } uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?sort_by=id_ascending
3594	2024-12-20T23:04:28.139Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42603 req_id = 7f50231d-0f6a-474d-b276-eda8023b6cc9 resource = Database result = Ok(()) uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?sort_by=id_ascending
3595	2024-12-20T23:04:28.141Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2916 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42603 req_id = 7f50231d-0f6a-474d-b276-eda8023b6cc9 response_code = 200 uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?sort_by=id_ascending
3596	2024-12-20T23:04:28.141Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42603 req_id = 9c31cd9f-4640-40e1-a0c0-6db43963414a roles = RoleSet { roles: {} } uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmM2I2MGJkMy02ZTM0LTQxNDAtYTc1NS01N2YyNmRiYzViMzQifX0%3D
3597	2024-12-20T23:04:28.141Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42603 req_id = 9c31cd9f-4640-40e1-a0c0-6db43963414a resource = Database result = Ok(()) uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmM2I2MGJkMy02ZTM0LTQxNDAtYTc1NS01N2YyNmRiYzViMzQifX0%3D
3598	2024-12-20T23:04:28.143Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2155 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42603 req_id = 9c31cd9f-4640-40e1-a0c0-6db43963414a response_code = 200 uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmM2I2MGJkMy02ZTM0LTQxNDAtYTc1NS01N2YyNmRiYzViMzQifX0%3D
3599	2024-12-20T23:04:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 22 reason = Timeout
3600	2024-12-20T23:04:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 22 reason = Timeout
3601	2024-12-20T23:04:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3602	2024-12-20T23:04:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 22 reason = Timeout
3603	2024-12-20T23:04:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
3604	2024-12-20T23:04:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
3605	2024-12-20T23:04:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 22 reason = Timeout
3606	2024-12-20T23:04:32.385Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
3607	2024-12-20T23:04:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 22 reason = Timeout
3608	2024-12-20T23:04:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
3609	2024-12-20T23:04:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 22 reason = Timeout
3610	2024-12-20T23:04:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3611	2024-12-20T23:04:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
3612	2024-12-20T23:04:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 22 reason = Timeout
3613	2024-12-20T23:04:32.385Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
3614	2024-12-20T23:04:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
3615	2024-12-20T23:04:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 64 reason = Timeout
3616	2024-12-20T23:04:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3617	2024-12-20T23:04:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 64 reason = Timeout
3618	2024-12-20T23:04:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3619	2024-12-20T23:04:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 22 reason = Timeout
3620	2024-12-20T23:04:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
3621	2024-12-20T23:04:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 24 reason = Timeout
3622	2024-12-20T23:04:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3623	2024-12-20T23:04:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 22 reason = Timeout
3624	2024-12-20T23:04:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
3625	2024-12-20T23:04:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 22 reason = Timeout
3626	2024-12-20T23:04:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3627	2024-12-20T23:04:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 22 reason = Timeout
3628	2024-12-20T23:04:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 22 reason = Timeout
3629	2024-12-20T23:04:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
3630	2024-12-20T23:04:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 22 reason = Timeout
3631	2024-12-20T23:04:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3632	2024-12-20T23:04:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 22 reason = Timeout
3633	2024-12-20T23:04:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3634	2024-12-20T23:04:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3635	2024-12-20T23:04:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
3636	2024-12-20T23:04:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
3637	2024-12-20T23:04:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
3638	2024-12-20T23:04:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3639	2024-12-20T23:04:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3640	2024-12-20T23:04:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
3641	2024-12-20T23:04:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
3642	2024-12-20T23:04:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
3643	2024-12-20T23:04:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3644	2024-12-20T23:04:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3645	2024-12-20T23:04:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
3646	2024-12-20T23:04:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3647	2024-12-20T23:04:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3648	2024-12-20T23:04:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
3649	2024-12-20T23:04:32.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3650	2024-12-20T23:04:32.388Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3651	2024-12-20T23:04:32.389Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1287, tv_nsec: 778845251 } })
3652	2024-12-20T23:04:32.389Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
3653	2024-12-20T23:04:32.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
3654	2024-12-20T23:04:32.390Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3655	2024-12-20T23:04:32.390Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.273508ms iteration = 22 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
3656	2024-12-20T23:04:32.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3657	2024-12-20T23:04:32.390Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3658	2024-12-20T23:04:32.392Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
3659	2024-12-20T23:04:32.392Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3660	2024-12-20T23:04:32.392Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3661	2024-12-20T23:04:32.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3662	2024-12-20T23:04:32.393Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3663	2024-12-20T23:04:32.394Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3664	2024-12-20T23:04:32.395Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3665	2024-12-20T23:04:32.395Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3666	2024-12-20T23:04:32.395Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
3667	2024-12-20T23:04:32.395Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3668	2024-12-20T23:04:32.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3669	2024-12-20T23:04:32.403Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1287, tv_nsec: 792723722 } })
3670	2024-12-20T23:04:32.403Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
3671	2024-12-20T23:04:32.404Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3672	2024-12-20T23:04:32.408Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3673	2024-12-20T23:04:32.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3674	2024-12-20T23:04:32.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
3675	2024-12-20T23:04:32.408Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3676	2024-12-20T23:04:32.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3677	2024-12-20T23:04:32.411Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3678	2024-12-20T23:04:32.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3679	2024-12-20T23:04:32.414Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1287, tv_nsec: 803681923 } })
3680	2024-12-20T23:04:32.414Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
3681	2024-12-20T23:04:32.415Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3682	2024-12-20T23:04:32.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3683	2024-12-20T23:04:32.417Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3684	2024-12-20T23:04:32.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
3685	2024-12-20T23:04:32.421Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3686	2024-12-20T23:04:32.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3687	2024-12-20T23:04:32.432Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
3688	2024-12-20T23:04:32.432Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 46.315702ms iteration = 64 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
3689	2024-12-20T23:04:32.432Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
3690	2024-12-20T23:04:32.433Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
3691	2024-12-20T23:04:32.434Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
3692	2024-12-20T23:04:32.434Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
3693	2024-12-20T23:04:32.436Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 50.764134ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
3694	2024-12-20T23:04:32.436Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 50.872207ms iteration = 22 status = {"errors":[],"finish_invoked_ok":[]}
3695	2024-12-20T23:04:32.437Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3696	2024-12-20T23:04:32.437Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
3697	2024-12-20T23:04:32.437Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3698	2024-12-20T23:04:32.437Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
3699	2024-12-20T23:04:32.438Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
3700	2024-12-20T23:04:32.438Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
3701	2024-12-20T23:04:32.438Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
3702	2024-12-20T23:04:32.438Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
3703	2024-12-20T23:04:32.439Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3704	2024-12-20T23:04:32.439Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3705	2024-12-20T23:04:32.439Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3706	2024-12-20T23:04:32.441Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
3707	2024-12-20T23:04:32.441Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
3708	2024-12-20T23:04:32.441Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 55.492074ms iteration = 22 status = {"errors":[],"garbage_collect_requested":[]}
3709	2024-12-20T23:04:32.441Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3710	2024-12-20T23:04:32.441Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
3711	2024-12-20T23:04:32.441Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3712	2024-12-20T23:04:32.442Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3713	2024-12-20T23:04:32.442Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3714	2024-12-20T23:04:32.442Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3715	2024-12-20T23:04:32.444Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 58.470206ms iteration = 64 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
3716	2024-12-20T23:04:32.445Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 287d6774-d3e6-4b73-8c77-c90534d4a54d, lookup_type: ById(287d6774-d3e6-4b73-8c77-c90534d4a54d) } result = Ok(())
3717	2024-12-20T23:04:32.445Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3718	2024-12-20T23:04:32.445Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3719	2024-12-20T23:04:32.446Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 61.113889ms iteration = 22 status = 0
3720	2024-12-20T23:04:32.448Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3721	2024-12-20T23:04:32.448Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3722	2024-12-20T23:04:32.450Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
3723	2024-12-20T23:04:32.450Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3724	2024-12-20T23:04:32.451Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
3725	2024-12-20T23:04:32.451Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3726	2024-12-20T23:04:32.451Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3727	2024-12-20T23:04:32.451Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3728	2024-12-20T23:04:32.465Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
3729	2024-12-20T23:04:32.465Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3730	2024-12-20T23:04:32.466Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4d88439b-6ec0-4e69-baf4-44accaf2bf1a", "content-length": "26", "date": "Fri, 20 Dec 2024 23:04:32 GMT"} })
3731	2024-12-20T23:04:32.466Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3732	2024-12-20T23:04:32.468Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3733	2024-12-20T23:04:32.469Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
3734	2024-12-20T23:04:32.469Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
3735	2024-12-20T23:04:32.472Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 86.895386ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
3736	2024-12-20T23:04:32.479Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
3737	2024-12-20T23:04:32.479Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 93.388404ms iteration = 22 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
3738	2024-12-20T23:04:32.479Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
3739	2024-12-20T23:04:32.479Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
3740	2024-12-20T23:04:32.479Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
3741	2024-12-20T23:04:32.479Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
3742	2024-12-20T23:04:32.483Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
3743	2024-12-20T23:04:32.483Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3744	2024-12-20T23:04:32.483Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b7d19c19-51bc-449f-b1d9-959a16a79266", "content-length": "26", "date": "Fri, 20 Dec 2024 23:04:32 GMT"} })
3745	2024-12-20T23:04:32.483Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3746	2024-12-20T23:04:32.486Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3747	2024-12-20T23:04:32.487Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 101.434356ms iteration = 24 status = {"physical_disks_added":0}
3748	2024-12-20T23:04:32.488Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 zone_address = fd00:1122:3344:101::2
3749	2024-12-20T23:04:32.488Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3750	2024-12-20T23:04:32.488Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c0f2d85-16fe-4796-9822-2b4c163ed6ae", "content-length": "26", "date": "Fri, 20 Dec 2024 23:04:32 GMT"} })
3751	2024-12-20T23:04:32.488Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3752	2024-12-20T23:04:32.490Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
3753	2024-12-20T23:04:32.500Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 115.028568ms iteration = 22 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
3754	2024-12-20T23:04:32.507Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
3755	2024-12-20T23:04:32.507Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
3756	2024-12-20T23:04:32.507Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 121.861856ms iteration = 22 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
3757	2024-12-20T23:04:32.540Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3758	2024-12-20T23:04:32.540Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3759	2024-12-20T23:04:32.541Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3760	2024-12-20T23:04:32.543Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
3761	2024-12-20T23:04:32.543Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3762	2024-12-20T23:04:32.543Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3763	2024-12-20T23:04:32.568Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 182.682996ms iteration = 22 status = {}
3764	2024-12-20T23:04:32.586Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
3765	2024-12-20T23:04:32.587Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "45d05c78-df67-4677-999d-39fa63ea555c", "content-length": "1", "date": "Fri, 20 Dec 2024 23:04:32 GMT"} })
3766	2024-12-20T23:04:32.588Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3767	2024-12-20T23:04:32.589Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3768	2024-12-20T23:04:32.608Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
3769	2024-12-20T23:04:32.608Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
3770	2024-12-20T23:04:32.608Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c6b7fe0-c404-429e-8a16-98f37d08c72a", "content-length": "372", "date": "Fri, 20 Dec 2024 23:04:32 GMT"} })
3771	2024-12-20T23:04:32.608Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3772	2024-12-20T23:04:32.609Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3773	2024-12-20T23:04:32.610Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3774	2024-12-20T23:04:32.610Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 224.851244ms iteration = 22 status = null
3775	2024-12-20T23:04:32.611Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
3776	2024-12-20T23:04:32.611Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3777	2024-12-20T23:04:32.612Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3778	2024-12-20T23:04:32.623Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
3779	2024-12-20T23:04:32.623Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
3780	2024-12-20T23:04:32.623Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "b18541dc-e0cc-4cf9-b864-0bd4ddd81d63", "date": "Fri, 20 Dec 2024 23:04:32 GMT"} })
3781	2024-12-20T23:04:32.623Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3782	2024-12-20T23:04:32.623Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3783	2024-12-20T23:04:32.625Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3784	2024-12-20T23:04:32.625Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3785	2024-12-20T23:04:32.628Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3786	2024-12-20T23:04:32.629Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3787	2024-12-20T23:04:32.631Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
3788	2024-12-20T23:04:32.631Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
3789	2024-12-20T23:04:32.632Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "4c3366c8-c6a9-4713-a02c-d8a46122982f", "date": "Fri, 20 Dec 2024 23:04:32 GMT"} })
3790	2024-12-20T23:04:32.634Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 248.559036ms iteration = 22 status = {}
3791	2024-12-20T23:04:32.660Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3792	2024-12-20T23:04:32.661Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3793	2024-12-20T23:04:32.663Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 03dd12a0-dc2d-46e1-a7f4-d3397b9b7aa7, rack_id: b394d0c6-19a8-4136-9148-af6c47c1f8e0, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(560ca89c-7c0c-4921-8e5c-b57676f695b0) } rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
3794	2024-12-20T23:04:32.663Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3795	2024-12-20T23:04:32.663Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3796	2024-12-20T23:04:32.678Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
3797	2024-12-20T23:04:32.679Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "519cfbd2-058b-4728-89f0-2fa2bf44a18c", "content-length": "134", "date": "Fri, 20 Dec 2024 23:04:32 GMT"} })
3798	2024-12-20T23:04:32.679Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
3799	2024-12-20T23:04:32.679Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 560ca89c-7c0c-4921-8e5c-b57676f695b0
3800	2024-12-20T23:04:32.679Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
3801	2024-12-20T23:04:32.679Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
3802	2024-12-20T23:04:32.679Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3803	2024-12-20T23:04:32.679Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3804	2024-12-20T23:04:32.679Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3805	2024-12-20T23:04:32.679Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
3806	2024-12-20T23:04:32.679Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3807	2024-12-20T23:04:32.679Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
3808	2024-12-20T23:04:32.680Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
3809	2024-12-20T23:04:32.680Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3810	2024-12-20T23:04:32.680Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3811	2024-12-20T23:04:32.682Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
3812	2024-12-20T23:04:32.682Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "21092c2e-9eac-43e2-9dda-8d7cddd1ca90", "content-length": "2", "date": "Fri, 20 Dec 2024 23:04:32 GMT"} })
3813	2024-12-20T23:04:32.682Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
3814	2024-12-20T23:04:32.683Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "c2489d0f-647e-4c4a-b9ad-9ab761b01150", "content-length": "52", "date": "Fri, 20 Dec 2024 23:04:32 GMT"} })
3815	2024-12-20T23:04:32.683Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
3816	2024-12-20T23:04:32.683Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
3817	2024-12-20T23:04:32.709Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
3818	2024-12-20T23:04:32.805Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "960697b9-dd97-4968-b327-51a4dc079fdd", "date": "Fri, 20 Dec 2024 23:04:32 GMT"} })
3819	2024-12-20T23:04:32.805Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch = Switch0
3820	2024-12-20T23:04:32.805Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
3821	2024-12-20T23:04:32.806Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "e1b049e3-4026-463c-b5aa-0fe13ad48c05", "content-length": "691", "date": "Fri, 20 Dec 2024 23:04:32 GMT"} })
3822	2024-12-20T23:04:32.806Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
3823	2024-12-20T23:04:32.806Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
3824	2024-12-20T23:04:32.806Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
3825	2024-12-20T23:04:32.806Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "51a150b5-b305-435f-9045-5d5dc8d34ba0", "content-length": "682", "date": "Fri, 20 Dec 2024 23:04:32 GMT"} })
3826	2024-12-20T23:04:32.806Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3827	2024-12-20T23:04:32.806Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3828	2024-12-20T23:04:32.808Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 422.788319ms iteration = 22 status = {}
3829	2024-12-20T23:04:32.809Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3830	2024-12-20T23:04:32.809Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3831	2024-12-20T23:04:32.813Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3832	2024-12-20T23:04:32.813Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3833	2024-12-20T23:04:32.815Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3834	2024-12-20T23:04:32.815Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3835	2024-12-20T23:04:32.823Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 437.954985ms iteration = 22 status = {}
3836	2024-12-20T23:04:33.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 308 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 0288e126-5b1d-40af-84fd-853d742fc7e4 response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
3837	2024-12-20T23:04:42.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 65 reason = Timeout
3838	2024-12-20T23:04:42.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3839	2024-12-20T23:04:42.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 65 reason = Timeout
3840	2024-12-20T23:04:42.385Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3841	2024-12-20T23:04:42.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3842	2024-12-20T23:04:42.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3843	2024-12-20T23:04:42.387Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3844	2024-12-20T23:04:42.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3845	2024-12-20T23:04:42.390Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3846	2024-12-20T23:04:42.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3847	2024-12-20T23:04:42.392Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3848	2024-12-20T23:04:42.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3849	2024-12-20T23:04:42.394Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3850	2024-12-20T23:04:42.397Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
3851	2024-12-20T23:04:42.397Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3852	2024-12-20T23:04:42.397Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3853	2024-12-20T23:04:42.406Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
3854	2024-12-20T23:04:42.406Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.265034ms iteration = 65 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
3855	2024-12-20T23:04:42.421Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3856	2024-12-20T23:04:42.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3857	2024-12-20T23:04:42.450Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 65.254132ms iteration = 65 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
3858	2024-12-20T23:04:43.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 286 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 67b4121f-df91-4a20-91b5-2aa100e7eee7 response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
3859	2024-12-20T23:04:52.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 66 reason = Timeout
3860	2024-12-20T23:04:52.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3861	2024-12-20T23:04:52.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 66 reason = Timeout
3862	2024-12-20T23:04:52.385Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3863	2024-12-20T23:04:52.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3864	2024-12-20T23:04:52.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3865	2024-12-20T23:04:52.386Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3866	2024-12-20T23:04:52.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3867	2024-12-20T23:04:52.387Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3868	2024-12-20T23:04:52.388Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.526867ms iteration = 66 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
3869	2024-12-20T23:04:52.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3870	2024-12-20T23:04:52.389Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3871	2024-12-20T23:04:52.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3872	2024-12-20T23:04:52.391Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3873	2024-12-20T23:04:52.391Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3874	2024-12-20T23:04:52.393Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3875	2024-12-20T23:04:52.395Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
3876	2024-12-20T23:04:52.395Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3877	2024-12-20T23:04:52.395Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3878	2024-12-20T23:04:52.406Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
3879	2024-12-20T23:04:52.406Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.102579ms iteration = 66 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
3880	2024-12-20T23:04:53.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 269 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = fe73350f-d237-4601-94a1-4c2b2edf19d9 response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
3881	2024-12-20T23:04:58.139Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42603 req_id = be5709e3-52cb-4b01-a5f6-a537031c8636 roles = RoleSet { roles: {} } uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?sort_by=id_ascending
3882	2024-12-20T23:04:58.139Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42603 req_id = be5709e3-52cb-4b01-a5f6-a537031c8636 resource = Database result = Ok(()) uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?sort_by=id_ascending
3883	2024-12-20T23:04:58.142Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3194 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42603 req_id = be5709e3-52cb-4b01-a5f6-a537031c8636 response_code = 200 uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?sort_by=id_ascending
3884	2024-12-20T23:04:58.142Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42603 req_id = 0869b65e-4414-4845-9595-54a33e50c4dc roles = RoleSet { roles: {} } uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmM2I2MGJkMy02ZTM0LTQxNDAtYTc1NS01N2YyNmRiYzViMzQifX0%3D
3885	2024-12-20T23:04:58.142Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42603 req_id = 0869b65e-4414-4845-9595-54a33e50c4dc resource = Database result = Ok(()) uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmM2I2MGJkMy02ZTM0LTQxNDAtYTc1NS01N2YyNmRiYzViMzQifX0%3D
3886	2024-12-20T23:04:58.145Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2875 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42603 req_id = 0869b65e-4414-4845-9595-54a33e50c4dc response_code = 200 uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmM2I2MGJkMy02ZTM0LTQxNDAtYTc1NS01N2YyNmRiYzViMzQifX0%3D
3887	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 23 reason = Timeout
3888	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3889	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 23 reason = Timeout
3890	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 23 reason = Timeout
3891	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
3892	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
3893	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 12 reason = Timeout
3894	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 12 reason = Timeout
3895	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
3896	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 12 reason = Timeout
3897	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
3898	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 23 reason = Timeout
3899	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
3900	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 23 reason = Timeout
3901	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
3902	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 23 reason = Timeout
3903	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
3904	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 12 reason = Timeout
3905	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
3906	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 23 reason = Timeout
3907	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3908	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
3909	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = external_endpoints iteration = 12 reason = Timeout
3910	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 12 reason = Timeout
3911	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3912	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 12 reason = Timeout
3913	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
3914	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 23 reason = Timeout
3915	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 23 reason = Timeout
3916	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 12 reason = Timeout
3917	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
3918	2024-12-20T23:05:02.385Z	INFO	nexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-20 22:55:02.385478041 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
3919	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
3920	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 23 reason = Timeout
3921	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 67 reason = Timeout
3922	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3923	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3924	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 23 reason = Timeout
3925	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 12 reason = Timeout
3926	2024-12-20T23:05:02.385Z	INFO	nexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
3927	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
3928	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 14 reason = Timeout
3929	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3930	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 25 reason = Timeout
3931	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3932	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 23 reason = Timeout
3933	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
3934	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
3935	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 13 reason = Timeout
3936	2024-12-20T23:05:02.385Z	WARN	nexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7
3937	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 23 reason = Timeout
3938	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 14.01\u{b5}s iteration = 13 status = {"enabled":false,"target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7"}
3939	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3940	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 12 reason = Timeout
3941	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
3942	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3943	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 67 reason = Timeout
3944	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3945	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 14 reason = Timeout
3946	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 23 reason = Timeout
3947	2024-12-20T23:05:02.385Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
3948	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3949	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
3950	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3951	2024-12-20T23:05:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
3952	2024-12-20T23:05:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
3953	2024-12-20T23:05:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
3954	2024-12-20T23:05:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
3955	2024-12-20T23:05:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
3956	2024-12-20T23:05:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
3957	2024-12-20T23:05:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3958	2024-12-20T23:05:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
3959	2024-12-20T23:05:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
3960	2024-12-20T23:05:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
3961	2024-12-20T23:05:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3962	2024-12-20T23:05:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
3963	2024-12-20T23:05:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3964	2024-12-20T23:05:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
3965	2024-12-20T23:05:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3966	2024-12-20T23:05:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
3967	2024-12-20T23:05:02.386Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("616ce87c-1f39-49d9-ba35-ee63b2650ecd.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b4615881-2f68-4c82-be17-071bf3ae3641.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c9217e92-0af3-4930-acb2-aa82ded5b782.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("c9217e92-0af3-4930-acb2-aa82ded5b782.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 1317, tv_nsec: 775804917 } })
3968	2024-12-20T23:05:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3969	2024-12-20T23:05:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3970	2024-12-20T23:05:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3971	2024-12-20T23:05:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3972	2024-12-20T23:05:02.386Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1e296b8d-6800-4056-8034-a1ec2d005fff.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("ace0592c-7755-46f8-b47a-c9b6444fe64c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("ace0592c-7755-46f8-b47a-c9b6444fe64c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1317, tv_nsec: 775865309 } })
3973	2024-12-20T23:05:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3974	2024-12-20T23:05:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3975	2024-12-20T23:05:02.386Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
3976	2024-12-20T23:05:02.386Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.415049ms iteration = 12 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
3977	2024-12-20T23:05:02.386Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
3978	2024-12-20T23:05:02.386Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.327287ms iteration = 12 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
3979	2024-12-20T23:05:02.388Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.689883ms iteration = 23 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
3980	2024-12-20T23:05:02.388Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 3.292631ms iteration = 23 status = {"errors":[],"garbage_collect_requested":[]}
3981	2024-12-20T23:05:02.388Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1317, tv_nsec: 778418449 } })
3982	2024-12-20T23:05:02.388Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
3983	2024-12-20T23:05:02.389Z	DEBG	nexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
3984	2024-12-20T23:05:02.389Z	INFO	nexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
3985	2024-12-20T23:05:02.389Z	DEBG	nexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 4.251997ms iteration = 12 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
3986	2024-12-20T23:05:02.389Z	DEBG	nexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 4.317398ms iteration = 12 status = {"errors":[],"found_port_ok":[]}
3987	2024-12-20T23:05:02.390Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3988	2024-12-20T23:05:02.390Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3989	2024-12-20T23:05:02.391Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1317, tv_nsec: 780712102 } })
3990	2024-12-20T23:05:02.391Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
3991	2024-12-20T23:05:02.391Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3992	2024-12-20T23:05:02.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
3993	2024-12-20T23:05:02.392Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
3994	2024-12-20T23:05:02.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
3995	2024-12-20T23:05:02.392Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
3996	2024-12-20T23:05:02.392Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3997	2024-12-20T23:05:02.393Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3998	2024-12-20T23:05:02.393Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3999	2024-12-20T23:05:02.393Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4000	2024-12-20T23:05:02.393Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
4001	2024-12-20T23:05:02.393Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
4002	2024-12-20T23:05:02.393Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
4003	2024-12-20T23:05:02.395Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
4004	2024-12-20T23:05:02.395Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4005	2024-12-20T23:05:02.395Z	INFO	nexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-20 22:55:02.385478041 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
4006	2024-12-20T23:05:02.395Z	DEBG	nexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 10.492628ms iteration = 12 status = {"expiration":"2024-12-20T22:55:02.385478041Z","pruned":[]}
4007	2024-12-20T23:05:02.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
4008	2024-12-20T23:05:02.396Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4009	2024-12-20T23:05:02.396Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
4010	2024-12-20T23:05:02.396Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
4011	2024-12-20T23:05:02.396Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
4012	2024-12-20T23:05:02.397Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
4013	2024-12-20T23:05:02.397Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4014	2024-12-20T23:05:02.399Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
4015	2024-12-20T23:05:02.399Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4016	2024-12-20T23:05:02.399Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
4017	2024-12-20T23:05:02.399Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4018	2024-12-20T23:05:02.399Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4019	2024-12-20T23:05:02.399Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4020	2024-12-20T23:05:02.400Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4021	2024-12-20T23:05:02.400Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4022	2024-12-20T23:05:02.401Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
4023	2024-12-20T23:05:02.401Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4024	2024-12-20T23:05:02.401Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4025	2024-12-20T23:05:02.403Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4026	2024-12-20T23:05:02.403Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4027	2024-12-20T23:05:02.406Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
4028	2024-12-20T23:05:02.406Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
4029	2024-12-20T23:05:02.406Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
4030	2024-12-20T23:05:02.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
4031	2024-12-20T23:05:02.407Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4032	2024-12-20T23:05:02.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4033	2024-12-20T23:05:02.407Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1317, tv_nsec: 797042190 } })
4034	2024-12-20T23:05:02.407Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
4035	2024-12-20T23:05:02.408Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4036	2024-12-20T23:05:02.408Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4037	2024-12-20T23:05:02.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4038	2024-12-20T23:05:02.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
4039	2024-12-20T23:05:02.411Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4040	2024-12-20T23:05:02.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4041	2024-12-20T23:05:02.425Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
4042	2024-12-20T23:05:02.425Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 40.508832ms iteration = 23 status = {"errors":[],"finish_invoked_ok":[]}
4043	2024-12-20T23:05:02.425Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
4044	2024-12-20T23:05:02.429Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
4045	2024-12-20T23:05:02.429Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4046	2024-12-20T23:05:02.429Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
4047	2024-12-20T23:05:02.429Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4048	2024-12-20T23:05:02.432Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
4049	2024-12-20T23:05:02.432Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
4050	2024-12-20T23:05:02.433Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 48.368609ms iteration = 67 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
4051	2024-12-20T23:05:02.434Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 48.692026ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
4052	2024-12-20T23:05:02.437Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4053	2024-12-20T23:05:02.438Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4054	2024-12-20T23:05:02.440Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
4055	2024-12-20T23:05:02.440Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4056	2024-12-20T23:05:02.440Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
4057	2024-12-20T23:05:02.440Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
4058	2024-12-20T23:05:02.441Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
4059	2024-12-20T23:05:02.441Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4060	2024-12-20T23:05:02.441Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4061	2024-12-20T23:05:02.443Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4062	2024-12-20T23:05:02.443Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4063	2024-12-20T23:05:02.443Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4064	2024-12-20T23:05:02.443Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4065	2024-12-20T23:05:02.444Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4066	2024-12-20T23:05:02.444Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 58.915838ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
4067	2024-12-20T23:05:02.445Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4068	2024-12-20T23:05:02.445Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4069	2024-12-20T23:05:02.446Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4070	2024-12-20T23:05:02.446Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4071	2024-12-20T23:05:02.446Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4072	2024-12-20T23:05:02.446Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
4073	2024-12-20T23:05:02.446Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4074	2024-12-20T23:05:02.447Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4075	2024-12-20T23:05:02.450Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4076	2024-12-20T23:05:02.451Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4077	2024-12-20T23:05:02.454Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4078	2024-12-20T23:05:02.455Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
4079	2024-12-20T23:05:02.455Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 69.967021ms iteration = 23 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
4080	2024-12-20T23:05:02.455Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
4081	2024-12-20T23:05:02.456Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
4082	2024-12-20T23:05:02.456Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4083	2024-12-20T23:05:02.456Z	DEBG	nexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
4084	2024-12-20T23:05:02.456Z	INFO	nexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
4085	2024-12-20T23:05:02.456Z	DEBG	nexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 70.795854ms iteration = 12 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
4086	2024-12-20T23:05:02.456Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4087	2024-12-20T23:05:02.456Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4088	2024-12-20T23:05:02.457Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 287d6774-d3e6-4b73-8c77-c90534d4a54d, lookup_type: ById(287d6774-d3e6-4b73-8c77-c90534d4a54d) } result = Ok(())
4089	2024-12-20T23:05:02.457Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4090	2024-12-20T23:05:02.457Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4091	2024-12-20T23:05:02.459Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
4092	2024-12-20T23:05:02.459Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4093	2024-12-20T23:05:02.459Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4094	2024-12-20T23:05:02.461Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4095	2024-12-20T23:05:02.461Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4096	2024-12-20T23:05:02.467Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
4097	2024-12-20T23:05:02.467Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
4098	2024-12-20T23:05:02.467Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 82.033253ms iteration = 23 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
4099	2024-12-20T23:05:02.477Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 91.749809ms iteration = 23 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
4100	2024-12-20T23:05:02.480Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
4101	2024-12-20T23:05:02.481Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
4102	2024-12-20T23:05:02.481Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
4103	2024-12-20T23:05:02.481Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4104	2024-12-20T23:05:02.481Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4105	2024-12-20T23:05:02.489Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
4106	2024-12-20T23:05:02.489Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
4107	2024-12-20T23:05:02.489Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
4108	2024-12-20T23:05:02.489Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4109	2024-12-20T23:05:02.489Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4110	2024-12-20T23:05:02.516Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4111	2024-12-20T23:05:02.516Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4112	2024-12-20T23:05:02.520Z	WARN	nexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
4113	2024-12-20T23:05:02.521Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 135.555112ms iteration = 12 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
4114	2024-12-20T23:05:02.546Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
4115	2024-12-20T23:05:02.546Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4116	2024-12-20T23:05:02.546Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a735a04-da01-4870-a8cc-c1264d310d77", "content-length": "26", "date": "Fri, 20 Dec 2024 23:05:02 GMT"} })
4117	2024-12-20T23:05:02.546Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4118	2024-12-20T23:05:02.549Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
4119	2024-12-20T23:05:02.565Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
4120	2024-12-20T23:05:02.636Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
4121	2024-12-20T23:05:02.690Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
4122	2024-12-20T23:05:02.690Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4123	2024-12-20T23:05:02.691Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3af5e693-cd3b-4f6a-880a-fb19192b9719", "content-length": "26", "date": "Fri, 20 Dec 2024 23:05:02 GMT"} })
4124	2024-12-20T23:05:02.691Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4125	2024-12-20T23:05:02.693Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
4126	2024-12-20T23:05:02.714Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
4127	2024-12-20T23:05:02.715Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "d50d867d-5fc0-480e-a95e-a45b54e55886", "content-length": "1", "date": "Fri, 20 Dec 2024 23:05:02 GMT"} })
4128	2024-12-20T23:05:02.717Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4129	2024-12-20T23:05:02.717Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4130	2024-12-20T23:05:02.732Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 zone_address = fd00:1122:3344:101::2
4131	2024-12-20T23:05:02.732Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4132	2024-12-20T23:05:02.732Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e21b7a6-b3eb-42ff-9650-92a186cde183", "content-length": "26", "date": "Fri, 20 Dec 2024 23:05:02 GMT"} })
4133	2024-12-20T23:05:02.732Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4134	2024-12-20T23:05:02.734Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
4135	2024-12-20T23:05:02.758Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
4136	2024-12-20T23:05:02.786Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
4137	2024-12-20T23:05:02.786Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
4138	2024-12-20T23:05:02.787Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "c19baa2b-be6c-4e11-b5b7-5483f229898b", "content-length": "372", "date": "Fri, 20 Dec 2024 23:05:02 GMT"} })
4139	2024-12-20T23:05:02.787Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4140	2024-12-20T23:05:02.787Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4141	2024-12-20T23:05:02.790Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 404.718233ms iteration = 23 status = 0
4142	2024-12-20T23:05:02.790Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4143	2024-12-20T23:05:02.791Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4144	2024-12-20T23:05:02.793Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
4145	2024-12-20T23:05:02.793Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4146	2024-12-20T23:05:02.793Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4147	2024-12-20T23:05:02.795Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 409.844704ms iteration = 23 status = null
4148	2024-12-20T23:05:02.795Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
4149	2024-12-20T23:05:02.795Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4150	2024-12-20T23:05:02.795Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4151	2024-12-20T23:05:02.803Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
4152	2024-12-20T23:05:02.803Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 418.056389ms iteration = 67 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
4153	2024-12-20T23:05:02.813Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
4154	2024-12-20T23:05:02.813Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
4155	2024-12-20T23:05:02.816Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "9fa3bc33-8bb9-4525-9f4d-3d1bd5636e79", "date": "Fri, 20 Dec 2024 23:05:02 GMT"} })
4156	2024-12-20T23:05:02.816Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4157	2024-12-20T23:05:02.817Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4158	2024-12-20T23:05:02.823Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4159	2024-12-20T23:05:02.823Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4160	2024-12-20T23:05:02.827Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4161	2024-12-20T23:05:02.827Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4162	2024-12-20T23:05:02.830Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 444.923556ms iteration = 23 status = {}
4163	2024-12-20T23:05:02.830Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4164	2024-12-20T23:05:02.831Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
4165	2024-12-20T23:05:02.831Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
4166	2024-12-20T23:05:02.831Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "09a153f6-3c78-4192-9867-24af9e7689c7", "date": "Fri, 20 Dec 2024 23:05:02 GMT"} })
4167	2024-12-20T23:05:02.833Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
4168	2024-12-20T23:05:02.833Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4169	2024-12-20T23:05:02.833Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 447.898269ms iteration = 23 status = {}
4170	2024-12-20T23:05:02.833Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4171	2024-12-20T23:05:02.836Z	DEBG	nexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-20T22:53:58.073373Z, time_modified: 2024-12-20T22:53:58.073373Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, db_silo: Silo { identity: SiloIdentity { id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-20T22:54:39.847015Z, time_modified: 2024-12-20T22:54:39.847015Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, db_silo: Silo { identity: SiloIdentity { id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-20T22:54:39.847015Z, time_modified: 2024-12-20T22:54:39.847015Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156" }] }) }
4172	2024-12-20T23:05:02.836Z	DEBG	nexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 451.026334ms iteration = 12 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"ee579470-b16b-4af1-ad3a-b3d29f9a2fb2","tls_certs":["ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156"]}},"default_endpoint":{"silo_id":"ee579470-b16b-4af1-ad3a-b3d29f9a2fb2","tls_certs":["ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
4173	2024-12-20T23:05:02.838Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
4174	2024-12-20T23:05:02.838Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 5fa20c62-75c0-4531-be07-51b87d303d18 dns_zone_name = oxide.test found_so_far = 0 version = 2
4175	2024-12-20T23:05:02.838Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4176	2024-12-20T23:05:02.842Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4177	2024-12-20T23:05:02.843Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4178	2024-12-20T23:05:02.847Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
4179	2024-12-20T23:05:02.847Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4180	2024-12-20T23:05:02.847Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4181	2024-12-20T23:05:02.847Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
4182	2024-12-20T23:05:02.848Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "c0d87dfe-0fbc-4b56-9c7a-1974c878221d", "date": "Fri, 20 Dec 2024 23:05:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4183	2024-12-20T23:05:02.848Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4184	2024-12-20T23:05:02.849Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 5fa20c62-75c0-4531-be07-51b87d303d18 dns_zone_name = oxide.test found_so_far = 1 version = 2
4185	2024-12-20T23:05:02.849Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
4186	2024-12-20T23:05:02.849Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-20 22:54:39.899407 UTC dns_group = external generation = 2
4187	2024-12-20T23:05:02.849Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 464.040142ms iteration = 12 status = {"generation":2}
4188	2024-12-20T23:05:02.850Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "6a67f4b5-7f26-46d7-9676-682d63ed8d52", "date": "Fri, 20 Dec 2024 23:05:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4189	2024-12-20T23:05:02.850Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4190	2024-12-20T23:05:02.852Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4191	2024-12-20T23:05:02.852Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 466.970582ms iteration = 14 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
4192	2024-12-20T23:05:02.874Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
4193	2024-12-20T23:05:02.874Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 1e86bf23-6ffd-4a12-93c7-0061d7d17590 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
4194	2024-12-20T23:05:02.874Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4195	2024-12-20T23:05:02.874Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4196	2024-12-20T23:05:02.880Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
4197	2024-12-20T23:05:02.880Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "fdf054af-4beb-4a2e-ae86-281f087548ad", "date": "Fri, 20 Dec 2024 23:05:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
4198	2024-12-20T23:05:02.880Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
4199	2024-12-20T23:05:02.882Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "641f64f8-808e-4871-a405-321a285a96af", "date": "Fri, 20 Dec 2024 23:05:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
4200	2024-12-20T23:05:02.882Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
4201	2024-12-20T23:05:02.884Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "dc5f0d6b-2d6d-429d-8908-c2fc761a2ab0", "date": "Fri, 20 Dec 2024 23:05:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
4202	2024-12-20T23:05:02.884Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
4203	2024-12-20T23:05:02.885Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
4204	2024-12-20T23:05:02.885Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 500.210595ms iteration = 14 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
4205	2024-12-20T23:05:02.893Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 507.885785ms iteration = 25 status = {"physical_disks_added":0}
4206	2024-12-20T23:05:02.901Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4207	2024-12-20T23:05:02.901Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4208	2024-12-20T23:05:02.909Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4209	2024-12-20T23:05:02.911Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
4210	2024-12-20T23:05:02.911Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4211	2024-12-20T23:05:02.911Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4212	2024-12-20T23:05:02.914Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 1e86bf23-6ffd-4a12-93c7-0061d7d17590 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
4213	2024-12-20T23:05:02.914Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
4214	2024-12-20T23:05:02.914Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-20 22:54:38.956071 UTC dns_group = internal generation = 1
4215	2024-12-20T23:05:02.914Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 528.983325ms iteration = 12 status = {"generation":1}
4216	2024-12-20T23:05:02.926Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4217	2024-12-20T23:05:02.926Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4218	2024-12-20T23:05:02.929Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 03dd12a0-dc2d-46e1-a7f4-d3397b9b7aa7, rack_id: b394d0c6-19a8-4136-9148-af6c47c1f8e0, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(560ca89c-7c0c-4921-8e5c-b57676f695b0) } rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
4219	2024-12-20T23:05:02.929Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4220	2024-12-20T23:05:02.929Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4221	2024-12-20T23:05:02.931Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4222	2024-12-20T23:05:02.933Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
4223	2024-12-20T23:05:02.933Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4224	2024-12-20T23:05:02.933Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4225	2024-12-20T23:05:02.934Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4226	2024-12-20T23:05:02.936Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
4227	2024-12-20T23:05:02.936Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4228	2024-12-20T23:05:02.936Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4229	2024-12-20T23:05:02.937Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
4230	2024-12-20T23:05:02.937Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "1798249b-da5d-4af7-9d14-cc264acd5563", "content-length": "134", "date": "Fri, 20 Dec 2024 23:05:02 GMT"} })
4231	2024-12-20T23:05:02.937Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
4232	2024-12-20T23:05:02.937Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 560ca89c-7c0c-4921-8e5c-b57676f695b0
4233	2024-12-20T23:05:02.938Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
4234	2024-12-20T23:05:02.938Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
4235	2024-12-20T23:05:02.938Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4236	2024-12-20T23:05:02.938Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4237	2024-12-20T23:05:02.938Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4238	2024-12-20T23:05:02.938Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
4239	2024-12-20T23:05:02.938Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4240	2024-12-20T23:05:02.938Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
4241	2024-12-20T23:05:02.938Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
4242	2024-12-20T23:05:02.938Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4243	2024-12-20T23:05:02.939Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4244	2024-12-20T23:05:02.941Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
4245	2024-12-20T23:05:02.941Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "18c539b9-bffb-494d-ba5d-9bb498ff3d8c", "content-length": "2", "date": "Fri, 20 Dec 2024 23:05:02 GMT"} })
4246	2024-12-20T23:05:02.941Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
4247	2024-12-20T23:05:02.941Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc21505a-665d-47fc-8784-dd487facd4d0", "content-length": "52", "date": "Fri, 20 Dec 2024 23:05:02 GMT"} })
4248	2024-12-20T23:05:02.941Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
4249	2024-12-20T23:05:02.941Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
4250	2024-12-20T23:05:03.025Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
4251	2024-12-20T23:05:03.051Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "3487148c-da70-4654-80af-17e6a3c6db16", "content-length": "691", "date": "Fri, 20 Dec 2024 23:05:02 GMT"} })
4252	2024-12-20T23:05:03.051Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "607bc971-bb12-43a8-a9ad-344936b7d9db", "date": "Fri, 20 Dec 2024 23:05:02 GMT"} })
4253	2024-12-20T23:05:03.051Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch = Switch0
4254	2024-12-20T23:05:03.051Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
4255	2024-12-20T23:05:03.051Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
4256	2024-12-20T23:05:03.051Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
4257	2024-12-20T23:05:03.051Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
4258	2024-12-20T23:05:03.051Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "7356683d-11ea-4392-a79d-0d16a617dd8b", "content-length": "682", "date": "Fri, 20 Dec 2024 23:05:02 GMT"} })
4259	2024-12-20T23:05:03.051Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4260	2024-12-20T23:05:03.052Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4261	2024-12-20T23:05:03.052Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 667.345194ms iteration = 23 status = {}
4262	2024-12-20T23:05:03.056Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4263	2024-12-20T23:05:03.056Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4264	2024-12-20T23:05:03.059Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4265	2024-12-20T23:05:03.060Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4266	2024-12-20T23:05:03.062Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4267	2024-12-20T23:05:03.062Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4268	2024-12-20T23:05:03.073Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 687.740325ms iteration = 23 status = {}
4269	2024-12-20T23:05:03.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 303 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = fa619b03-74c8-4226-bbb6-eec7a0d784a0 response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
4270	2024-12-20T23:05:03.241Z	DEBG	nexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
4271	2024-12-20T23:05:03.241Z	DEBG	nexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
4272	2024-12-20T23:05:03.271Z	DEBG	nexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
4273	2024-12-20T23:05:03.272Z	INFO	nexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:63897
4274	2024-12-20T23:05:03.272Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:63897 req_id = 6ff5260e-f4cb-427e-a43f-7f730a084130 roles = RoleSet { roles: {} } uri = /metrics/producers
4275	2024-12-20T23:05:03.272Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:63897 req_id = 6ff5260e-f4cb-427e-a43f-7f730a084130 resource = Database result = Ok(()) uri = /metrics/producers
4276	2024-12-20T23:05:03.307Z	INFO	nexus: assigned collector to new producer collector_id = 103fc383-10f6-4327-a1b1-dabac4e1e365 file = nexus/src/app/oximeter.rs:76 producer_id = f3b60bd3-6e34-4140-a755-57f26dbc5b34
4277	2024-12-20T23:05:03.307Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 35171 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:63897 req_id = 6ff5260e-f4cb-427e-a43f-7f730a084130 response_code = 201 uri = /metrics/producers
4278	2024-12-20T23:05:03.307Z	DEBG	nexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "6ff5260e-f4cb-427e-a43f-7f730a084130", "content-length": "41", "date": "Fri, 20 Dec 2024 23:05:03 GMT"} })
4279	2024-12-20T23:05:03.308Z	DEBG	nexus (producer-registration-task): registered with nexus successfully
4280	2024-12-20T23:05:03.308Z	DEBG	nexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
4281	2024-12-20T23:05:12.384Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 68 reason = Timeout
4282	2024-12-20T23:05:12.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 68 reason = Timeout
4283	2024-12-20T23:05:12.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4284	2024-12-20T23:05:12.384Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4285	2024-12-20T23:05:12.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4286	2024-12-20T23:05:12.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4287	2024-12-20T23:05:12.414Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4288	2024-12-20T23:05:12.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4289	2024-12-20T23:05:12.414Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4290	2024-12-20T23:05:12.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4291	2024-12-20T23:05:12.417Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4292	2024-12-20T23:05:12.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4293	2024-12-20T23:05:12.419Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4294	2024-12-20T23:05:12.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4295	2024-12-20T23:05:12.420Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4296	2024-12-20T23:05:12.422Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
4297	2024-12-20T23:05:12.422Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4298	2024-12-20T23:05:12.422Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4299	2024-12-20T23:05:12.433Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
4300	2024-12-20T23:05:12.433Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 48.682937ms iteration = 68 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
4301	2024-12-20T23:05:12.443Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 59.318539ms iteration = 68 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
4302	2024-12-20T23:05:13.138Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42603 req_id = 702905e4-e698-4ff6-a26a-db54e8b292a6 roles = RoleSet { roles: {} } uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?sort_by=id_ascending
4303	2024-12-20T23:05:13.139Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42603 req_id = 702905e4-e698-4ff6-a26a-db54e8b292a6 resource = Database result = Ok(()) uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?sort_by=id_ascending
4304	2024-12-20T23:05:13.142Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3567 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42603 req_id = 702905e4-e698-4ff6-a26a-db54e8b292a6 response_code = 200 uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?sort_by=id_ascending
4305	2024-12-20T23:05:13.142Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42603 req_id = c1007e25-6812-46aa-9398-9b3e9b134dd4 roles = RoleSet { roles: {} } uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmM2I2MGJkMy02ZTM0LTQxNDAtYTc1NS01N2YyNmRiYzViMzQifX0%3D
4306	2024-12-20T23:05:13.142Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42603 req_id = c1007e25-6812-46aa-9398-9b3e9b134dd4 resource = Database result = Ok(()) uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmM2I2MGJkMy02ZTM0LTQxNDAtYTc1NS01N2YyNmRiYzViMzQifX0%3D
4307	2024-12-20T23:05:13.144Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2231 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42603 req_id = c1007e25-6812-46aa-9398-9b3e9b134dd4 response_code = 200 uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmM2I2MGJkMy02ZTM0LTQxNDAtYTc1NS01N2YyNmRiYzViMzQifX0%3D
4308	2024-12-20T23:05:13.149Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 439 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 2cf0545a-8887-49c0-85f4-e837dce615d2 response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
4309	2024-12-20T23:05:22.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 69 reason = Timeout
4310	2024-12-20T23:05:22.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 69 reason = Timeout
4311	2024-12-20T23:05:22.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4312	2024-12-20T23:05:22.385Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4313	2024-12-20T23:05:22.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4314	2024-12-20T23:05:22.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4315	2024-12-20T23:05:22.387Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4316	2024-12-20T23:05:22.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4317	2024-12-20T23:05:22.390Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4318	2024-12-20T23:05:22.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4319	2024-12-20T23:05:22.392Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4320	2024-12-20T23:05:22.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4321	2024-12-20T23:05:22.415Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4322	2024-12-20T23:05:22.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4323	2024-12-20T23:05:22.420Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4324	2024-12-20T23:05:22.423Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
4325	2024-12-20T23:05:22.423Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4326	2024-12-20T23:05:22.423Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4327	2024-12-20T23:05:22.434Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
4328	2024-12-20T23:05:22.434Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 48.579863ms iteration = 69 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
4329	2024-12-20T23:05:22.443Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 58.28029ms iteration = 69 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
4330	2024-12-20T23:05:23.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 272 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = f9314163-4e5e-4d7e-854f-85cfa1f34af8 response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
4331	2024-12-20T23:05:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 24 reason = Timeout
4332	2024-12-20T23:05:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 24 reason = Timeout
4333	2024-12-20T23:05:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
4334	2024-12-20T23:05:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4335	2024-12-20T23:05:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 24 reason = Timeout
4336	2024-12-20T23:05:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 24 reason = Timeout
4337	2024-12-20T23:05:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
4338	2024-12-20T23:05:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
4339	2024-12-20T23:05:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 24 reason = Timeout
4340	2024-12-20T23:05:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
4341	2024-12-20T23:05:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 24 reason = Timeout
4342	2024-12-20T23:05:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 24 reason = Timeout
4343	2024-12-20T23:05:32.384Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
4344	2024-12-20T23:05:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
4345	2024-12-20T23:05:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 24 reason = Timeout
4346	2024-12-20T23:05:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 24 reason = Timeout
4347	2024-12-20T23:05:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4348	2024-12-20T23:05:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 24 reason = Timeout
4349	2024-12-20T23:05:32.384Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
4350	2024-12-20T23:05:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4351	2024-12-20T23:05:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 70 reason = Timeout
4352	2024-12-20T23:05:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4353	2024-12-20T23:05:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4354	2024-12-20T23:05:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 70 reason = Timeout
4355	2024-12-20T23:05:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4356	2024-12-20T23:05:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 26 reason = Timeout
4357	2024-12-20T23:05:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 24 reason = Timeout
4358	2024-12-20T23:05:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4359	2024-12-20T23:05:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4360	2024-12-20T23:05:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 24 reason = Timeout
4361	2024-12-20T23:05:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
4362	2024-12-20T23:05:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 24 reason = Timeout
4363	2024-12-20T23:05:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
4364	2024-12-20T23:05:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 24 reason = Timeout
4365	2024-12-20T23:05:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
4366	2024-12-20T23:05:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4367	2024-12-20T23:05:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
4368	2024-12-20T23:05:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
4369	2024-12-20T23:05:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
4370	2024-12-20T23:05:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4371	2024-12-20T23:05:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
4372	2024-12-20T23:05:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4373	2024-12-20T23:05:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4374	2024-12-20T23:05:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
4375	2024-12-20T23:05:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4376	2024-12-20T23:05:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4377	2024-12-20T23:05:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
4378	2024-12-20T23:05:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
4379	2024-12-20T23:05:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4380	2024-12-20T23:05:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
4381	2024-12-20T23:05:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4382	2024-12-20T23:05:32.388Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1347, tv_nsec: 778113660 } })
4383	2024-12-20T23:05:32.388Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
4384	2024-12-20T23:05:32.388Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.923208ms iteration = 24 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
4385	2024-12-20T23:05:32.388Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4386	2024-12-20T23:05:32.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
4387	2024-12-20T23:05:32.390Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
4388	2024-12-20T23:05:32.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
4389	2024-12-20T23:05:32.394Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1347, tv_nsec: 784333581 } })
4390	2024-12-20T23:05:32.395Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
4391	2024-12-20T23:05:32.396Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4392	2024-12-20T23:05:32.399Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
4393	2024-12-20T23:05:32.399Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4394	2024-12-20T23:05:32.399Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4395	2024-12-20T23:05:32.403Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1347, tv_nsec: 792763963 } })
4396	2024-12-20T23:05:32.403Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
4397	2024-12-20T23:05:32.427Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
4398	2024-12-20T23:05:32.427Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
4399	2024-12-20T23:05:32.427Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
4400	2024-12-20T23:05:32.427Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4401	2024-12-20T23:05:32.427Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
4402	2024-12-20T23:05:32.428Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
4403	2024-12-20T23:05:32.428Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
4404	2024-12-20T23:05:32.429Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 44.281816ms iteration = 24 status = {"errors":[],"garbage_collect_requested":[]}
4405	2024-12-20T23:05:32.430Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
4406	2024-12-20T23:05:32.430Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4407	2024-12-20T23:05:32.430Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4408	2024-12-20T23:05:32.431Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4409	2024-12-20T23:05:32.433Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4410	2024-12-20T23:05:32.433Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4411	2024-12-20T23:05:32.433Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4412	2024-12-20T23:05:32.434Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
4413	2024-12-20T23:05:32.434Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
4414	2024-12-20T23:05:32.435Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4415	2024-12-20T23:05:32.435Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4416	2024-12-20T23:05:32.435Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4417	2024-12-20T23:05:32.435Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
4418	2024-12-20T23:05:32.435Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4419	2024-12-20T23:05:32.435Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4420	2024-12-20T23:05:32.436Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4421	2024-12-20T23:05:32.437Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4422	2024-12-20T23:05:32.438Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
4423	2024-12-20T23:05:32.438Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4424	2024-12-20T23:05:32.438Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4425	2024-12-20T23:05:32.439Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
4426	2024-12-20T23:05:32.439Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4427	2024-12-20T23:05:32.440Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4428	2024-12-20T23:05:32.442Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
4429	2024-12-20T23:05:32.442Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
4430	2024-12-20T23:05:32.444Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4431	2024-12-20T23:05:32.444Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4432	2024-12-20T23:05:32.444Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 59.881915ms iteration = 24 status = {"errors":[],"finish_invoked_ok":[]}
4433	2024-12-20T23:05:32.444Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4434	2024-12-20T23:05:32.445Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4435	2024-12-20T23:05:32.445Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4436	2024-12-20T23:05:32.445Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4437	2024-12-20T23:05:32.446Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4438	2024-12-20T23:05:32.448Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
4439	2024-12-20T23:05:32.448Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4440	2024-12-20T23:05:32.448Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4441	2024-12-20T23:05:32.449Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
4442	2024-12-20T23:05:32.449Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 65.150899ms iteration = 24 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
4443	2024-12-20T23:05:32.463Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
4444	2024-12-20T23:05:32.464Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
4445	2024-12-20T23:05:32.467Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4446	2024-12-20T23:05:32.468Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4447	2024-12-20T23:05:32.470Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4448	2024-12-20T23:05:32.470Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4449	2024-12-20T23:05:32.479Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
4450	2024-12-20T23:05:32.479Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4451	2024-12-20T23:05:32.479Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd769131-5991-4e04-87ac-17780a0b9d4c", "content-length": "26", "date": "Fri, 20 Dec 2024 23:05:32 GMT"} })
4452	2024-12-20T23:05:32.479Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4453	2024-12-20T23:05:32.480Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
4454	2024-12-20T23:05:32.480Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
4455	2024-12-20T23:05:32.480Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
4456	2024-12-20T23:05:32.480Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 95.881852ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
4457	2024-12-20T23:05:32.480Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 95.715109ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
4458	2024-12-20T23:05:32.480Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
4459	2024-12-20T23:05:32.482Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 97.631181ms iteration = 70 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
4460	2024-12-20T23:05:32.482Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
4461	2024-12-20T23:05:32.489Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
4462	2024-12-20T23:05:32.489Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4463	2024-12-20T23:05:32.489Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "45486289-a1f9-42fd-ae71-ce56697bba6d", "content-length": "26", "date": "Fri, 20 Dec 2024 23:05:32 GMT"} })
4464	2024-12-20T23:05:32.489Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4465	2024-12-20T23:05:32.491Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
4466	2024-12-20T23:05:32.495Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4467	2024-12-20T23:05:32.498Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
4468	2024-12-20T23:05:32.498Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4469	2024-12-20T23:05:32.498Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4470	2024-12-20T23:05:32.503Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 zone_address = fd00:1122:3344:101::2
4471	2024-12-20T23:05:32.503Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4472	2024-12-20T23:05:32.503Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ddd0fd7e-51f6-4dc8-9b50-c48cc2988489", "content-length": "26", "date": "Fri, 20 Dec 2024 23:05:32 GMT"} })
4473	2024-12-20T23:05:32.503Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4474	2024-12-20T23:05:32.504Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 120.013365ms iteration = 24 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
4475	2024-12-20T23:05:32.505Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
4476	2024-12-20T23:05:32.510Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
4477	2024-12-20T23:05:32.510Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 125.669361ms iteration = 70 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
4478	2024-12-20T23:05:32.512Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4479	2024-12-20T23:05:32.512Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4480	2024-12-20T23:05:32.514Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4481	2024-12-20T23:05:32.517Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 287d6774-d3e6-4b73-8c77-c90534d4a54d, lookup_type: ById(287d6774-d3e6-4b73-8c77-c90534d4a54d) } result = Ok(())
4482	2024-12-20T23:05:32.517Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4483	2024-12-20T23:05:32.517Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4484	2024-12-20T23:05:32.519Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 134.502133ms iteration = 24 status = 0
4485	2024-12-20T23:05:32.520Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
4486	2024-12-20T23:05:32.520Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
4487	2024-12-20T23:05:32.520Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 135.867501ms iteration = 24 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
4488	2024-12-20T23:05:32.555Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 170.188154ms iteration = 26 status = {"physical_disks_added":0}
4489	2024-12-20T23:05:32.604Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4490	2024-12-20T23:05:32.604Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4491	2024-12-20T23:05:32.606Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4492	2024-12-20T23:05:32.607Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
4493	2024-12-20T23:05:32.607Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
4494	2024-12-20T23:05:32.607Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4495	2024-12-20T23:05:32.607Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4496	2024-12-20T23:05:32.607Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "caa6a24b-d315-48c8-aa80-b8e2897ad56d", "content-length": "1", "date": "Fri, 20 Dec 2024 23:05:32 GMT"} })
4497	2024-12-20T23:05:32.609Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4498	2024-12-20T23:05:32.609Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4499	2024-12-20T23:05:32.609Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4500	2024-12-20T23:05:32.611Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
4501	2024-12-20T23:05:32.611Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4502	2024-12-20T23:05:32.611Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4503	2024-12-20T23:05:32.660Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 276.089751ms iteration = 24 status = {}
4504	2024-12-20T23:05:32.679Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
4505	2024-12-20T23:05:32.679Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
4506	2024-12-20T23:05:32.680Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "92a0d5e3-410c-4e69-a0c1-f4cffdfab175", "content-length": "372", "date": "Fri, 20 Dec 2024 23:05:32 GMT"} })
4507	2024-12-20T23:05:32.680Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4508	2024-12-20T23:05:32.680Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4509	2024-12-20T23:05:32.681Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 296.574813ms iteration = 24 status = null
4510	2024-12-20T23:05:32.724Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {7af95d5c-cc00-4843-9aab-7ae751cf8054: {10.151.1.175: {b9f0e957-0765-4023-bc13-350c229e799c}, 10.151.1.176: {b9f0e957-0765-4023-bc13-350c229e799c}}} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
4511	2024-12-20T23:05:32.724Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
4512	2024-12-20T23:05:32.724Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "3d645fa3-81de-4040-b178-fd31d7d568cb", "date": "Fri, 20 Dec 2024 23:05:32 GMT"} })
4513	2024-12-20T23:05:32.724Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4514	2024-12-20T23:05:32.725Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4515	2024-12-20T23:05:32.727Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4516	2024-12-20T23:05:32.727Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4517	2024-12-20T23:05:32.730Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4518	2024-12-20T23:05:32.731Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4519	2024-12-20T23:05:32.751Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
4520	2024-12-20T23:05:32.752Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba0c7928-fc4d-4c15-9b6d-8d12bb2c2ede", "content-length": "810", "date": "Fri, 20 Dec 2024 23:05:32 GMT"} })
4521	2024-12-20T23:05:32.752Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {VirtualNetworkInterfaceHost { physical_host_ip: fd00:1122:3344:101::1, virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), vni: Vni(9502392) }} sled = buskin
4522	2024-12-20T23:05:32.752Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
4523	2024-12-20T23:05:32.752Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
4524	2024-12-20T23:05:32.754Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 369.939968ms iteration = 24 status = {}
4525	2024-12-20T23:05:32.755Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
4526	2024-12-20T23:05:32.756Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
4527	2024-12-20T23:05:32.762Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "dacaae79-af03-447f-9f69-3d47b97adf51", "date": "Fri, 20 Dec 2024 23:05:32 GMT"} })
4528	2024-12-20T23:05:32.764Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4529	2024-12-20T23:05:32.765Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4530	2024-12-20T23:05:32.766Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 381.369542ms iteration = 24 status = {}
4531	2024-12-20T23:05:32.769Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 03dd12a0-dc2d-46e1-a7f4-d3397b9b7aa7, rack_id: b394d0c6-19a8-4136-9148-af6c47c1f8e0, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(560ca89c-7c0c-4921-8e5c-b57676f695b0) } rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
4532	2024-12-20T23:05:32.769Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4533	2024-12-20T23:05:32.769Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4534	2024-12-20T23:05:32.781Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
4535	2024-12-20T23:05:32.782Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "325affd4-9a8b-4d8c-a528-5838920980b8", "content-length": "134", "date": "Fri, 20 Dec 2024 23:05:32 GMT"} })
4536	2024-12-20T23:05:32.782Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
4537	2024-12-20T23:05:32.782Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 560ca89c-7c0c-4921-8e5c-b57676f695b0
4538	2024-12-20T23:05:32.782Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
4539	2024-12-20T23:05:32.782Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
4540	2024-12-20T23:05:32.782Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4541	2024-12-20T23:05:32.782Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4542	2024-12-20T23:05:32.782Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4543	2024-12-20T23:05:32.782Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
4544	2024-12-20T23:05:32.783Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4545	2024-12-20T23:05:32.783Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
4546	2024-12-20T23:05:32.783Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
4547	2024-12-20T23:05:32.783Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4548	2024-12-20T23:05:32.783Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4549	2024-12-20T23:05:32.785Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
4550	2024-12-20T23:05:32.786Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e2b8f62-bb70-415c-b56c-14f1d0b1b96e", "content-length": "2", "date": "Fri, 20 Dec 2024 23:05:32 GMT"} })
4551	2024-12-20T23:05:32.786Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
4552	2024-12-20T23:05:32.786Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "e56a2ab7-6865-42fe-9f79-b238b13a95a9", "content-length": "52", "date": "Fri, 20 Dec 2024 23:05:32 GMT"} })
4553	2024-12-20T23:05:32.786Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
4554	2024-12-20T23:05:32.786Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
4555	2024-12-20T23:05:32.907Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "a5bc9b6a-4e15-4c84-aa54-8838ff2b721a", "date": "Fri, 20 Dec 2024 23:05:32 GMT"} })
4556	2024-12-20T23:05:32.907Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch = Switch0
4557	2024-12-20T23:05:32.907Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
4558	2024-12-20T23:05:32.907Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "a05e53ee-46a1-47ad-95f5-fb5e025e61f4", "content-length": "682", "date": "Fri, 20 Dec 2024 23:05:32 GMT"} })
4559	2024-12-20T23:05:32.907Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4560	2024-12-20T23:05:32.908Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4561	2024-12-20T23:05:32.911Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4562	2024-12-20T23:05:32.911Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4563	2024-12-20T23:05:32.915Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4564	2024-12-20T23:05:32.916Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4565	2024-12-20T23:05:32.917Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4566	2024-12-20T23:05:32.918Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4567	2024-12-20T23:05:32.926Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 541.439317ms iteration = 24 status = {}
4568	2024-12-20T23:05:33.149Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 354 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 9a832c00-de8a-4144-909d-b40a146236f0 response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
4569	2024-12-20T23:05:42.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 71 reason = Timeout
4570	2024-12-20T23:05:42.385Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4571	2024-12-20T23:05:42.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 71 reason = Timeout
4572	2024-12-20T23:05:42.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4573	2024-12-20T23:05:42.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4574	2024-12-20T23:05:42.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4575	2024-12-20T23:05:42.387Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4576	2024-12-20T23:05:42.388Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4577	2024-12-20T23:05:42.388Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4578	2024-12-20T23:05:42.389Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.475963ms iteration = 71 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
4579	2024-12-20T23:05:42.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4580	2024-12-20T23:05:42.390Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4581	2024-12-20T23:05:42.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4582	2024-12-20T23:05:42.392Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4583	2024-12-20T23:05:42.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4584	2024-12-20T23:05:42.394Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4585	2024-12-20T23:05:42.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
4586	2024-12-20T23:05:42.396Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4587	2024-12-20T23:05:42.397Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4588	2024-12-20T23:05:42.406Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
4589	2024-12-20T23:05:42.406Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.779181ms iteration = 71 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
4590	2024-12-20T23:05:43.138Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42603 req_id = cb3e1b15-3537-412a-b0f3-4088ac0bd891 roles = RoleSet { roles: {} } uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?sort_by=id_ascending
4591	2024-12-20T23:05:43.139Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42603 req_id = cb3e1b15-3537-412a-b0f3-4088ac0bd891 resource = Database result = Ok(()) uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?sort_by=id_ascending
4592	2024-12-20T23:05:43.142Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3866 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42603 req_id = cb3e1b15-3537-412a-b0f3-4088ac0bd891 response_code = 200 uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?sort_by=id_ascending
4593	2024-12-20T23:05:43.142Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42603 req_id = 6a8f841d-98fa-4dde-88f6-56cd711b145f roles = RoleSet { roles: {} } uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmM2I2MGJkMy02ZTM0LTQxNDAtYTc1NS01N2YyNmRiYzViMzQifX0%3D
4594	2024-12-20T23:05:43.143Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42603 req_id = 6a8f841d-98fa-4dde-88f6-56cd711b145f resource = Database result = Ok(()) uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmM2I2MGJkMy02ZTM0LTQxNDAtYTc1NS01N2YyNmRiYzViMzQifX0%3D
4595	2024-12-20T23:05:43.145Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2984 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42603 req_id = 6a8f841d-98fa-4dde-88f6-56cd711b145f response_code = 200 uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmM2I2MGJkMy02ZTM0LTQxNDAtYTc1NS01N2YyNmRiYzViMzQifX0%3D
4596	2024-12-20T23:05:43.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 263 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 77fb6850-bde5-4601-b515-3974108cf893 response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
4597	2024-12-20T23:05:52.384Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 72 reason = Timeout
4598	2024-12-20T23:05:52.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 72 reason = Timeout
4599	2024-12-20T23:05:52.384Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4600	2024-12-20T23:05:52.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4601	2024-12-20T23:05:52.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4602	2024-12-20T23:05:52.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4603	2024-12-20T23:05:52.386Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4604	2024-12-20T23:05:52.387Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4605	2024-12-20T23:05:52.387Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4606	2024-12-20T23:05:52.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4607	2024-12-20T23:05:52.389Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4608	2024-12-20T23:05:52.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4609	2024-12-20T23:05:52.391Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4610	2024-12-20T23:05:52.391Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4611	2024-12-20T23:05:52.392Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4612	2024-12-20T23:05:52.394Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
4613	2024-12-20T23:05:52.394Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4614	2024-12-20T23:05:52.395Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4615	2024-12-20T23:05:52.409Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
4616	2024-12-20T23:05:52.409Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.165721ms iteration = 72 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
4617	2024-12-20T23:05:52.421Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 37.11433ms iteration = 72 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
4618	2024-12-20T23:05:53.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 280 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = deea5c99-315a-4a7e-9b1a-39687ff48c18 response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
4619	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 25 reason = Timeout
4620	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 25 reason = Timeout
4621	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
4622	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
4623	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
4624	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 6 reason = Timeout
4625	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 25 reason = Timeout
4626	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
4627	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 25 reason = Timeout
4628	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4629	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 27 reason = Timeout
4630	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4631	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 73 reason = Timeout
4632	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4633	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4634	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 25 reason = Timeout
4635	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 13 reason = Timeout
4636	2024-12-20T23:06:02.385Z	INFO	nexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
4637	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
4638	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
4639	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 25 reason = Timeout
4640	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = external_endpoints iteration = 13 reason = Timeout
4641	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4642	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 73 reason = Timeout
4643	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4644	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4645	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 25 reason = Timeout
4646	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 25 reason = Timeout
4647	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4648	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 25 reason = Timeout
4649	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 13 reason = Timeout
4650	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
4651	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
4652	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 15 reason = Timeout
4653	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 25 reason = Timeout
4654	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
4655	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 13 reason = Timeout
4656	2024-12-20T23:06:02.385Z	INFO	nexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-20 22:56:02.385473052 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
4657	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
4658	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 13 reason = Timeout
4659	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
4660	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4661	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 25 reason = Timeout
4662	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 15 reason = Timeout
4663	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
4664	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 14 reason = Timeout
4665	2024-12-20T23:06:02.385Z	WARN	nexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7
4666	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 13 reason = Timeout
4667	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 13.131\u{b5}s iteration = 14 status = {"enabled":false,"target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7"}
4668	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
4669	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4670	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 13 reason = Timeout
4671	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 25 reason = Timeout
4672	2024-12-20T23:06:02.385Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
4673	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4674	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 13 reason = Timeout
4675	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 13 reason = Timeout
4676	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
4677	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 25 reason = Timeout
4678	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
4679	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 25 reason = Timeout
4680	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
4681	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 13 reason = Timeout
4682	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
4683	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
4684	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
4685	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
4686	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
4687	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4688	2024-12-20T23:06:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4689	2024-12-20T23:06:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
4690	2024-12-20T23:06:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4691	2024-12-20T23:06:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4692	2024-12-20T23:06:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4693	2024-12-20T23:06:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4694	2024-12-20T23:06:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
4695	2024-12-20T23:06:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
4696	2024-12-20T23:06:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4697	2024-12-20T23:06:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4698	2024-12-20T23:06:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
4699	2024-12-20T23:06:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
4700	2024-12-20T23:06:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4701	2024-12-20T23:06:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4702	2024-12-20T23:06:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
4703	2024-12-20T23:06:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
4704	2024-12-20T23:06:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
4705	2024-12-20T23:06:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4706	2024-12-20T23:06:02.386Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1e296b8d-6800-4056-8034-a1ec2d005fff.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("ace0592c-7755-46f8-b47a-c9b6444fe64c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("ace0592c-7755-46f8-b47a-c9b6444fe64c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1377, tv_nsec: 775906890 } })
4707	2024-12-20T23:06:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
4708	2024-12-20T23:06:02.386Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
4709	2024-12-20T23:06:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
4710	2024-12-20T23:06:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
4711	2024-12-20T23:06:02.386Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("616ce87c-1f39-49d9-ba35-ee63b2650ecd.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b4615881-2f68-4c82-be17-071bf3ae3641.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c9217e92-0af3-4930-acb2-aa82ded5b782.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("c9217e92-0af3-4930-acb2-aa82ded5b782.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 1377, tv_nsec: 775929430 } })
4712	2024-12-20T23:06:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
4713	2024-12-20T23:06:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
4714	2024-12-20T23:06:02.386Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
4715	2024-12-20T23:06:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
4716	2024-12-20T23:06:02.386Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
4717	2024-12-20T23:06:02.386Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
4718	2024-12-20T23:06:02.386Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.034658ms iteration = 13 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
4719	2024-12-20T23:06:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
4720	2024-12-20T23:06:02.386Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
4721	2024-12-20T23:06:02.386Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.148631ms iteration = 13 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
4722	2024-12-20T23:06:02.388Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1377, tv_nsec: 778521022 } })
4723	2024-12-20T23:06:02.389Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
4724	2024-12-20T23:06:02.389Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4725	2024-12-20T23:06:02.389Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.497824ms iteration = 25 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
4726	2024-12-20T23:06:02.390Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4727	2024-12-20T23:06:02.390Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4728	2024-12-20T23:06:02.391Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4729	2024-12-20T23:06:02.391Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
4730	2024-12-20T23:06:02.391Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
4731	2024-12-20T23:06:02.391Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4732	2024-12-20T23:06:02.391Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
4733	2024-12-20T23:06:02.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
4734	2024-12-20T23:06:02.392Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4735	2024-12-20T23:06:02.393Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4736	2024-12-20T23:06:02.393Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4737	2024-12-20T23:06:02.394Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4738	2024-12-20T23:06:02.394Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4739	2024-12-20T23:06:02.395Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4740	2024-12-20T23:06:02.395Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4741	2024-12-20T23:06:02.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4742	2024-12-20T23:06:02.396Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4743	2024-12-20T23:06:02.396Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4744	2024-12-20T23:06:02.398Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
4745	2024-12-20T23:06:02.398Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
4746	2024-12-20T23:06:02.398Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
4747	2024-12-20T23:06:02.398Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4748	2024-12-20T23:06:02.398Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
4749	2024-12-20T23:06:02.398Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4750	2024-12-20T23:06:02.399Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
4751	2024-12-20T23:06:02.399Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4752	2024-12-20T23:06:02.399Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4753	2024-12-20T23:06:02.401Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4754	2024-12-20T23:06:02.401Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1377, tv_nsec: 791665723 } })
4755	2024-12-20T23:06:02.402Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
4756	2024-12-20T23:06:02.402Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4757	2024-12-20T23:06:02.402Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4758	2024-12-20T23:06:02.402Z	DEBG	nexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 16.937566ms iteration = 13 status = {"errors":[],"found_port_ok":[]}
4759	2024-12-20T23:06:02.402Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4760	2024-12-20T23:06:02.404Z	INFO	nexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-20 22:56:02.385473052 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
4761	2024-12-20T23:06:02.404Z	DEBG	nexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 18.728935ms iteration = 13 status = {"expiration":"2024-12-20T22:56:02.385473052Z","pruned":[]}
4762	2024-12-20T23:06:02.404Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
4763	2024-12-20T23:06:02.404Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4764	2024-12-20T23:06:02.404Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4765	2024-12-20T23:06:02.405Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
4766	2024-12-20T23:06:02.405Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
4767	2024-12-20T23:06:02.405Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
4768	2024-12-20T23:06:02.407Z	DEBG	nexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
4769	2024-12-20T23:06:02.407Z	INFO	nexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
4770	2024-12-20T23:06:02.407Z	DEBG	nexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 21.585573ms iteration = 13 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
4771	2024-12-20T23:06:02.407Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4772	2024-12-20T23:06:02.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
4773	2024-12-20T23:06:02.409Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
4774	2024-12-20T23:06:02.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
4775	2024-12-20T23:06:02.409Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4776	2024-12-20T23:06:02.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
4777	2024-12-20T23:06:02.412Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
4778	2024-12-20T23:06:02.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
4779	2024-12-20T23:06:02.414Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4780	2024-12-20T23:06:02.414Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4781	2024-12-20T23:06:02.415Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4782	2024-12-20T23:06:02.416Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4783	2024-12-20T23:06:02.416Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1377, tv_nsec: 806115590 } })
4784	2024-12-20T23:06:02.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4785	2024-12-20T23:06:02.416Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
4786	2024-12-20T23:06:02.417Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4787	2024-12-20T23:06:02.417Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4788	2024-12-20T23:06:02.420Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
4789	2024-12-20T23:06:02.425Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
4790	2024-12-20T23:06:02.425Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
4791	2024-12-20T23:06:02.425Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
4792	2024-12-20T23:06:02.425Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4793	2024-12-20T23:06:02.425Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4794	2024-12-20T23:06:02.425Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4795	2024-12-20T23:06:02.426Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
4796	2024-12-20T23:06:02.426Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4797	2024-12-20T23:06:02.426Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4798	2024-12-20T23:06:02.427Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4799	2024-12-20T23:06:02.427Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4800	2024-12-20T23:06:02.429Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
4801	2024-12-20T23:06:02.429Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 43.762792ms iteration = 25 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
4802	2024-12-20T23:06:02.430Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
4803	2024-12-20T23:06:02.430Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4804	2024-12-20T23:06:02.430Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4805	2024-12-20T23:06:02.432Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
4806	2024-12-20T23:06:02.432Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
4807	2024-12-20T23:06:02.432Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
4808	2024-12-20T23:06:02.432Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
4809	2024-12-20T23:06:02.432Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
4810	2024-12-20T23:06:02.432Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4811	2024-12-20T23:06:02.432Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4812	2024-12-20T23:06:02.433Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 48.144462ms iteration = 25 status = {"errors":[],"garbage_collect_requested":[]}
4813	2024-12-20T23:06:02.434Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4814	2024-12-20T23:06:02.435Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4815	2024-12-20T23:06:02.437Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
4816	2024-12-20T23:06:02.437Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4817	2024-12-20T23:06:02.437Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
4818	2024-12-20T23:06:02.437Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4819	2024-12-20T23:06:02.438Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4820	2024-12-20T23:06:02.438Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4821	2024-12-20T23:06:02.444Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
4822	2024-12-20T23:06:02.444Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
4823	2024-12-20T23:06:02.446Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4824	2024-12-20T23:06:02.449Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
4825	2024-12-20T23:06:02.449Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
4826	2024-12-20T23:06:02.451Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
4827	2024-12-20T23:06:02.451Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4828	2024-12-20T23:06:02.451Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4829	2024-12-20T23:06:02.453Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4830	2024-12-20T23:06:02.454Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4831	2024-12-20T23:06:02.454Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4832	2024-12-20T23:06:02.456Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 70.539126ms iteration = 25 status = {"errors":[],"finish_invoked_ok":[]}
4833	2024-12-20T23:06:02.456Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
4834	2024-12-20T23:06:02.456Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
4835	2024-12-20T23:06:02.456Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
4836	2024-12-20T23:06:02.462Z	DEBG	nexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
4837	2024-12-20T23:06:02.462Z	INFO	nexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
4838	2024-12-20T23:06:02.462Z	DEBG	nexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 77.049665ms iteration = 13 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
4839	2024-12-20T23:06:02.468Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
4840	2024-12-20T23:06:02.468Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 83.168774ms iteration = 73 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
4841	2024-12-20T23:06:02.470Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
4842	2024-12-20T23:06:02.470Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
4843	2024-12-20T23:06:02.472Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 87.160193ms iteration = 73 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
4844	2024-12-20T23:06:02.473Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4845	2024-12-20T23:06:02.477Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4846	2024-12-20T23:06:02.478Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
4847	2024-12-20T23:06:02.479Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
4848	2024-12-20T23:06:02.479Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
4849	2024-12-20T23:06:02.479Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
4850	2024-12-20T23:06:02.480Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4851	2024-12-20T23:06:02.480Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4852	2024-12-20T23:06:02.481Z	DEBG	nexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 95.933014ms iteration = 6 status = {"nsuccess":5}
4853	2024-12-20T23:06:02.482Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4854	2024-12-20T23:06:02.485Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
4855	2024-12-20T23:06:02.485Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4856	2024-12-20T23:06:02.485Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4857	2024-12-20T23:06:02.487Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
4858	2024-12-20T23:06:02.487Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 1e86bf23-6ffd-4a12-93c7-0061d7d17590 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
4859	2024-12-20T23:06:02.487Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4860	2024-12-20T23:06:02.487Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4861	2024-12-20T23:06:02.489Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4862	2024-12-20T23:06:02.489Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
4863	2024-12-20T23:06:02.490Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
4864	2024-12-20T23:06:02.493Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
4865	2024-12-20T23:06:02.493Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
4866	2024-12-20T23:06:02.493Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
4867	2024-12-20T23:06:02.493Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
4868	2024-12-20T23:06:02.493Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4869	2024-12-20T23:06:02.493Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
4870	2024-12-20T23:06:02.493Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4871	2024-12-20T23:06:02.494Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 109.066115ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
4872	2024-12-20T23:06:02.497Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 1e86bf23-6ffd-4a12-93c7-0061d7d17590 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
4873	2024-12-20T23:06:02.497Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
4874	2024-12-20T23:06:02.497Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-20 22:54:38.956071 UTC dns_group = internal generation = 1
4875	2024-12-20T23:06:02.497Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 111.787069ms iteration = 13 status = {"generation":1}
4876	2024-12-20T23:06:02.500Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 115.413329ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
4877	2024-12-20T23:06:02.502Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4878	2024-12-20T23:06:02.502Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4879	2024-12-20T23:06:02.503Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 118.35126ms iteration = 27 status = {"physical_disks_added":0}
4880	2024-12-20T23:06:02.505Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4881	2024-12-20T23:06:02.505Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4882	2024-12-20T23:06:02.507Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4883	2024-12-20T23:06:02.509Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 287d6774-d3e6-4b73-8c77-c90534d4a54d, lookup_type: ById(287d6774-d3e6-4b73-8c77-c90534d4a54d) } result = Ok(())
4884	2024-12-20T23:06:02.509Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4885	2024-12-20T23:06:02.510Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4886	2024-12-20T23:06:02.511Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 126.69967ms iteration = 25 status = 0
4887	2024-12-20T23:06:02.512Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
4888	2024-12-20T23:06:02.512Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
4889	2024-12-20T23:06:02.526Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
4890	2024-12-20T23:06:02.526Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
4891	2024-12-20T23:06:02.526Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
4892	2024-12-20T23:06:02.526Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4893	2024-12-20T23:06:02.526Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4894	2024-12-20T23:06:02.527Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4895	2024-12-20T23:06:02.530Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
4896	2024-12-20T23:06:02.530Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4897	2024-12-20T23:06:02.530Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4898	2024-12-20T23:06:02.534Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
4899	2024-12-20T23:06:02.534Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
4900	2024-12-20T23:06:02.534Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 149.030332ms iteration = 25 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
4901	2024-12-20T23:06:02.536Z	INFO	nexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-20T22:53:58.073373Z, time_modified: 2024-12-20T22:53:58.073373Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, db_silo: Silo { identity: SiloIdentity { id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-20T22:54:39.847015Z, time_modified: 2024-12-20T22:54:39.847015Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, db_silo: Silo { identity: SiloIdentity { id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-20T22:54:39.847015Z, time_modified: 2024-12-20T22:54:39.847015Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:94
4902	2024-12-20T23:06:02.536Z	DEBG	nexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 151.512941ms iteration = 13 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"ee579470-b16b-4af1-ad3a-b3d29f9a2fb2","tls_certs":["ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156"]}},"default_endpoint":{"silo_id":"ee579470-b16b-4af1-ad3a-b3d29f9a2fb2","tls_certs":["ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
4903	2024-12-20T23:06:02.537Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4904	2024-12-20T23:06:02.537Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4905	2024-12-20T23:06:02.550Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 164.847817ms iteration = 25 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
4906	2024-12-20T23:06:02.562Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 zone_address = fd00:1122:3344:101::2
4907	2024-12-20T23:06:02.562Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4908	2024-12-20T23:06:02.563Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d831b340-0a5a-4848-a484-26feb68b863c", "content-length": "26", "date": "Fri, 20 Dec 2024 23:06:02 GMT"} })
4909	2024-12-20T23:06:02.563Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4910	2024-12-20T23:06:02.563Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
4911	2024-12-20T23:06:02.563Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4912	2024-12-20T23:06:02.563Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "20bbce19-7a77-4c4a-811b-3db8b52586fb", "content-length": "26", "date": "Fri, 20 Dec 2024 23:06:02 GMT"} })
4913	2024-12-20T23:06:02.563Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4914	2024-12-20T23:06:02.567Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
4915	2024-12-20T23:06:02.567Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
4916	2024-12-20T23:06:02.567Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 5fa20c62-75c0-4531-be07-51b87d303d18 dns_zone_name = oxide.test found_so_far = 0 version = 2
4917	2024-12-20T23:06:02.567Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4918	2024-12-20T23:06:02.567Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4919	2024-12-20T23:06:02.568Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
4920	2024-12-20T23:06:02.569Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4921	2024-12-20T23:06:02.573Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
4922	2024-12-20T23:06:02.573Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4923	2024-12-20T23:06:02.574Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4924	2024-12-20T23:06:02.576Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 5fa20c62-75c0-4531-be07-51b87d303d18 dns_zone_name = oxide.test found_so_far = 1 version = 2
4925	2024-12-20T23:06:02.576Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
4926	2024-12-20T23:06:02.576Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-20 22:54:39.899407 UTC dns_group = external generation = 2
4927	2024-12-20T23:06:02.576Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 190.989124ms iteration = 13 status = {"generation":2}
4928	2024-12-20T23:06:02.587Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
4929	2024-12-20T23:06:02.587Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4930	2024-12-20T23:06:02.587Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9548f5e-f19e-47a8-ab77-df4e3fdc0b16", "content-length": "26", "date": "Fri, 20 Dec 2024 23:06:02 GMT"} })
4931	2024-12-20T23:06:02.587Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4932	2024-12-20T23:06:02.588Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
4933	2024-12-20T23:06:02.596Z	WARN	nexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
4934	2024-12-20T23:06:02.596Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 211.097717ms iteration = 13 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
4935	2024-12-20T23:06:02.621Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
4936	2024-12-20T23:06:02.621Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
4937	2024-12-20T23:06:02.622Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "1de1ba19-a4c5-4b89-8413-bd5ad40c7d6d", "content-length": "372", "date": "Fri, 20 Dec 2024 23:06:02 GMT"} })
4938	2024-12-20T23:06:02.622Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4939	2024-12-20T23:06:02.622Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4940	2024-12-20T23:06:02.636Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4941	2024-12-20T23:06:02.636Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4942	2024-12-20T23:06:02.638Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4943	2024-12-20T23:06:02.640Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
4944	2024-12-20T23:06:02.640Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4945	2024-12-20T23:06:02.640Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4946	2024-12-20T23:06:02.641Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4947	2024-12-20T23:06:02.643Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
4948	2024-12-20T23:06:02.643Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4949	2024-12-20T23:06:02.643Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4950	2024-12-20T23:06:02.664Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
4951	2024-12-20T23:06:02.664Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
4952	2024-12-20T23:06:02.665Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "1908a4f4-1629-473b-8ce6-544a53c32b29", "date": "Fri, 20 Dec 2024 23:06:02 GMT"} })
4953	2024-12-20T23:06:02.665Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4954	2024-12-20T23:06:02.665Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4955	2024-12-20T23:06:02.667Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4956	2024-12-20T23:06:02.668Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4957	2024-12-20T23:06:02.670Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4958	2024-12-20T23:06:02.670Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4959	2024-12-20T23:06:02.673Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
4960	2024-12-20T23:06:02.673Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
4961	2024-12-20T23:06:02.673Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "d811c253-95b8-4fc4-805e-3938ecbf47b4", "date": "Fri, 20 Dec 2024 23:06:02 GMT"} })
4962	2024-12-20T23:06:02.675Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 289.437857ms iteration = 25 status = {}
4963	2024-12-20T23:06:02.702Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
4964	2024-12-20T23:06:02.714Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
4965	2024-12-20T23:06:02.759Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
4966	2024-12-20T23:06:02.759Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "55c0584b-e649-4e49-a327-8a444c8b5341", "content-length": "2", "date": "Fri, 20 Dec 2024 23:06:02 GMT"} })
4967	2024-12-20T23:06:02.763Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4968	2024-12-20T23:06:02.764Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4969	2024-12-20T23:06:02.771Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 386.473712ms iteration = 25 status = null
4970	2024-12-20T23:06:02.783Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 398.78232ms iteration = 25 status = {}
4971	2024-12-20T23:06:02.824Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
4972	2024-12-20T23:06:02.825Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee024e62-6dd5-45a4-9594-3ea7174eb0f5", "content-length": "691", "date": "Fri, 20 Dec 2024 23:06:02 GMT"} })
4973	2024-12-20T23:06:02.825Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
4974	2024-12-20T23:06:02.825Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
4975	2024-12-20T23:06:02.825Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
4976	2024-12-20T23:06:02.826Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 441.448192ms iteration = 25 status = {}
4977	2024-12-20T23:06:02.951Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
4978	2024-12-20T23:06:02.951Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "7bcab1f8-5020-416b-8293-1cdf16ee1ad4", "date": "Fri, 20 Dec 2024 23:06:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4979	2024-12-20T23:06:02.952Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4980	2024-12-20T23:06:02.952Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4981	2024-12-20T23:06:02.953Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "c6b8d54d-6639-42f2-a713-d8c308d4731f", "date": "Fri, 20 Dec 2024 23:06:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4982	2024-12-20T23:06:02.953Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4983	2024-12-20T23:06:02.953Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4984	2024-12-20T23:06:02.954Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4985	2024-12-20T23:06:02.954Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 569.077686ms iteration = 15 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
4986	2024-12-20T23:06:02.954Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
4987	2024-12-20T23:06:02.955Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 03dd12a0-dc2d-46e1-a7f4-d3397b9b7aa7, rack_id: b394d0c6-19a8-4136-9148-af6c47c1f8e0, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(560ca89c-7c0c-4921-8e5c-b57676f695b0) } rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
4988	2024-12-20T23:06:02.955Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4989	2024-12-20T23:06:02.955Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4990	2024-12-20T23:06:02.966Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
4991	2024-12-20T23:06:02.966Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4f801d7-6170-4aa9-b33c-851f909ee231", "content-length": "134", "date": "Fri, 20 Dec 2024 23:06:02 GMT"} })
4992	2024-12-20T23:06:02.966Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
4993	2024-12-20T23:06:02.966Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 560ca89c-7c0c-4921-8e5c-b57676f695b0
4994	2024-12-20T23:06:02.966Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
4995	2024-12-20T23:06:02.966Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
4996	2024-12-20T23:06:02.966Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4997	2024-12-20T23:06:02.966Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4998	2024-12-20T23:06:02.966Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4999	2024-12-20T23:06:02.966Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
5000	2024-12-20T23:06:02.966Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5001	2024-12-20T23:06:02.966Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
5002	2024-12-20T23:06:02.967Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
5003	2024-12-20T23:06:02.967Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5004	2024-12-20T23:06:02.967Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5005	2024-12-20T23:06:02.969Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
5006	2024-12-20T23:06:02.969Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "dc89775a-6135-4e05-9d76-7cd7f0769984", "content-length": "2", "date": "Fri, 20 Dec 2024 23:06:02 GMT"} })
5007	2024-12-20T23:06:02.969Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
5008	2024-12-20T23:06:02.969Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "cdfca74d-b080-4af7-ac13-8d2e89f797cb", "content-length": "52", "date": "Fri, 20 Dec 2024 23:06:02 GMT"} })
5009	2024-12-20T23:06:02.969Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
5010	2024-12-20T23:06:02.969Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
5011	2024-12-20T23:06:02.990Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
5012	2024-12-20T23:06:02.991Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "7cf99936-3211-498b-91cf-15b6c5469dc9", "date": "Fri, 20 Dec 2024 23:06:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
5013	2024-12-20T23:06:02.991Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
5014	2024-12-20T23:06:02.992Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "2d1f2ac0-7409-481c-914a-40a77023f66e", "date": "Fri, 20 Dec 2024 23:06:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
5015	2024-12-20T23:06:02.992Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
5016	2024-12-20T23:06:02.993Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "209532b9-56b1-4534-a002-c535a1372614", "date": "Fri, 20 Dec 2024 23:06:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
5017	2024-12-20T23:06:02.993Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
5018	2024-12-20T23:06:02.994Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
5019	2024-12-20T23:06:02.995Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 609.570668ms iteration = 15 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
5020	2024-12-20T23:06:03.072Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "8ace91be-5147-4483-9bfe-aebace8d83c1", "date": "Fri, 20 Dec 2024 23:06:02 GMT"} })
5021	2024-12-20T23:06:03.072Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch = Switch0
5022	2024-12-20T23:06:03.072Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
5023	2024-12-20T23:06:03.073Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb588869-ebe6-497f-93a1-d6e1dd813c2e", "content-length": "682", "date": "Fri, 20 Dec 2024 23:06:02 GMT"} })
5024	2024-12-20T23:06:03.073Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5025	2024-12-20T23:06:03.073Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5026	2024-12-20T23:06:03.077Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5027	2024-12-20T23:06:03.077Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5028	2024-12-20T23:06:03.081Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5029	2024-12-20T23:06:03.081Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5030	2024-12-20T23:06:03.083Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5031	2024-12-20T23:06:03.083Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5032	2024-12-20T23:06:03.091Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 706.085068ms iteration = 25 status = {}
5033	2024-12-20T23:06:03.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 249 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = b4a5df2f-cf4f-4626-955d-3fc399ac09cb response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
5034	2024-12-20T23:06:12.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 74 reason = Timeout
5035	2024-12-20T23:06:12.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5036	2024-12-20T23:06:12.384Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 74 reason = Timeout
5037	2024-12-20T23:06:12.384Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5038	2024-12-20T23:06:12.384Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5039	2024-12-20T23:06:12.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5040	2024-12-20T23:06:12.386Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5041	2024-12-20T23:06:12.387Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5042	2024-12-20T23:06:12.387Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5043	2024-12-20T23:06:12.388Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.316318ms iteration = 74 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
5044	2024-12-20T23:06:12.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5045	2024-12-20T23:06:12.389Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5046	2024-12-20T23:06:12.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5047	2024-12-20T23:06:12.391Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5048	2024-12-20T23:06:12.391Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5049	2024-12-20T23:06:12.392Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5050	2024-12-20T23:06:12.395Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
5051	2024-12-20T23:06:12.395Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5052	2024-12-20T23:06:12.395Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5053	2024-12-20T23:06:12.406Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
5054	2024-12-20T23:06:12.406Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.158778ms iteration = 74 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
5055	2024-12-20T23:06:13.149Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 288 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = b1d719a4-553d-41fe-9e84-e1fb90023570 response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
5056	2024-12-20T23:06:22.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 75 reason = Timeout
5057	2024-12-20T23:06:22.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 75 reason = Timeout
5058	2024-12-20T23:06:22.385Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5059	2024-12-20T23:06:22.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5060	2024-12-20T23:06:22.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5061	2024-12-20T23:06:22.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5062	2024-12-20T23:06:22.387Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5063	2024-12-20T23:06:22.388Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5064	2024-12-20T23:06:22.388Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5065	2024-12-20T23:06:22.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5066	2024-12-20T23:06:22.390Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5067	2024-12-20T23:06:22.390Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.972116ms iteration = 75 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
5068	2024-12-20T23:06:22.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5069	2024-12-20T23:06:22.392Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5070	2024-12-20T23:06:22.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5071	2024-12-20T23:06:22.394Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5072	2024-12-20T23:06:22.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
5073	2024-12-20T23:06:22.396Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5074	2024-12-20T23:06:22.397Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5075	2024-12-20T23:06:22.405Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
5076	2024-12-20T23:06:22.405Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.534604ms iteration = 75 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
5077	2024-12-20T23:06:23.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 267 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = aaae5a88-94df-48ba-b979-b0797890b8b0 response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
5078	2024-12-20T23:06:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 26 reason = Timeout
5079	2024-12-20T23:06:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
5080	2024-12-20T23:06:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 26 reason = Timeout
5081	2024-12-20T23:06:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 26 reason = Timeout
5082	2024-12-20T23:06:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5083	2024-12-20T23:06:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5084	2024-12-20T23:06:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 26 reason = Timeout
5085	2024-12-20T23:06:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
5086	2024-12-20T23:06:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 26 reason = Timeout
5087	2024-12-20T23:06:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 26 reason = Timeout
5088	2024-12-20T23:06:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5089	2024-12-20T23:06:32.384Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
5090	2024-12-20T23:06:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5091	2024-12-20T23:06:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 26 reason = Timeout
5092	2024-12-20T23:06:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
5093	2024-12-20T23:06:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 76 reason = Timeout
5094	2024-12-20T23:06:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5095	2024-12-20T23:06:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 26 reason = Timeout
5096	2024-12-20T23:06:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
5097	2024-12-20T23:06:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 26 reason = Timeout
5098	2024-12-20T23:06:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5099	2024-12-20T23:06:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 26 reason = Timeout
5100	2024-12-20T23:06:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 26 reason = Timeout
5101	2024-12-20T23:06:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
5102	2024-12-20T23:06:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 26 reason = Timeout
5103	2024-12-20T23:06:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5104	2024-12-20T23:06:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 28 reason = Timeout
5105	2024-12-20T23:06:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 76 reason = Timeout
5106	2024-12-20T23:06:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5107	2024-12-20T23:06:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 26 reason = Timeout
5108	2024-12-20T23:06:32.384Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
5109	2024-12-20T23:06:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5110	2024-12-20T23:06:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 26 reason = Timeout
5111	2024-12-20T23:06:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5112	2024-12-20T23:06:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5113	2024-12-20T23:06:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5114	2024-12-20T23:06:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5115	2024-12-20T23:06:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
5116	2024-12-20T23:06:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
5117	2024-12-20T23:06:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
5118	2024-12-20T23:06:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
5119	2024-12-20T23:06:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5120	2024-12-20T23:06:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5121	2024-12-20T23:06:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
5122	2024-12-20T23:06:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
5123	2024-12-20T23:06:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5124	2024-12-20T23:06:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5125	2024-12-20T23:06:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5126	2024-12-20T23:06:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5127	2024-12-20T23:06:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5128	2024-12-20T23:06:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
5129	2024-12-20T23:06:32.387Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1407, tv_nsec: 777364120 } })
5130	2024-12-20T23:06:32.387Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
5131	2024-12-20T23:06:32.388Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5132	2024-12-20T23:06:32.388Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.880696ms iteration = 26 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
5133	2024-12-20T23:06:32.388Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5134	2024-12-20T23:06:32.390Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5135	2024-12-20T23:06:32.390Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5136	2024-12-20T23:06:32.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
5137	2024-12-20T23:06:32.392Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5138	2024-12-20T23:06:32.392Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
5139	2024-12-20T23:06:32.392Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
5140	2024-12-20T23:06:32.392Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5141	2024-12-20T23:06:32.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
5142	2024-12-20T23:06:32.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
5143	2024-12-20T23:06:32.393Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
5144	2024-12-20T23:06:32.393Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5145	2024-12-20T23:06:32.393Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5146	2024-12-20T23:06:32.394Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5147	2024-12-20T23:06:32.395Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5148	2024-12-20T23:06:32.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
5149	2024-12-20T23:06:32.396Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5150	2024-12-20T23:06:32.396Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1407, tv_nsec: 786457699 } })
5151	2024-12-20T23:06:32.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5152	2024-12-20T23:06:32.397Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
5153	2024-12-20T23:06:32.397Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
5154	2024-12-20T23:06:32.397Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5155	2024-12-20T23:06:32.397Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5156	2024-12-20T23:06:32.400Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5157	2024-12-20T23:06:32.400Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5158	2024-12-20T23:06:32.401Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5159	2024-12-20T23:06:32.401Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5160	2024-12-20T23:06:32.401Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5161	2024-12-20T23:06:32.404Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
5162	2024-12-20T23:06:32.404Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5163	2024-12-20T23:06:32.404Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5164	2024-12-20T23:06:32.405Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5165	2024-12-20T23:06:32.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
5166	2024-12-20T23:06:32.407Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5167	2024-12-20T23:06:32.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5168	2024-12-20T23:06:32.420Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 35.211607ms iteration = 26 status = {"errors":[],"garbage_collect_requested":[]}
5169	2024-12-20T23:06:32.426Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
5170	2024-12-20T23:06:32.426Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
5171	2024-12-20T23:06:32.426Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5172	2024-12-20T23:06:32.426Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5173	2024-12-20T23:06:32.427Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5174	2024-12-20T23:06:32.427Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5175	2024-12-20T23:06:32.427Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 43.058402ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
5176	2024-12-20T23:06:32.427Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5177	2024-12-20T23:06:32.428Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5178	2024-12-20T23:06:32.430Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
5179	2024-12-20T23:06:32.430Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5180	2024-12-20T23:06:32.430Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5181	2024-12-20T23:06:32.431Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 46.727232ms iteration = 26 status = {"errors":[],"finish_invoked_ok":[]}
5182	2024-12-20T23:06:32.431Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5183	2024-12-20T23:06:32.431Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5184	2024-12-20T23:06:32.432Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5185	2024-12-20T23:06:32.433Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5186	2024-12-20T23:06:32.433Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5187	2024-12-20T23:06:32.434Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5188	2024-12-20T23:06:32.435Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5189	2024-12-20T23:06:32.435Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5190	2024-12-20T23:06:32.435Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5191	2024-12-20T23:06:32.435Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 50.716992ms iteration = 76 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
5192	2024-12-20T23:06:32.435Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 50.832146ms iteration = 26 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
5193	2024-12-20T23:06:32.435Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1407, tv_nsec: 825455240 } })
5194	2024-12-20T23:06:32.436Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
5195	2024-12-20T23:06:32.436Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5196	2024-12-20T23:06:32.436Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
5197	2024-12-20T23:06:32.436Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5198	2024-12-20T23:06:32.437Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
5199	2024-12-20T23:06:32.437Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
5200	2024-12-20T23:06:32.437Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 52.386668ms iteration = 26 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
5201	2024-12-20T23:06:32.439Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
5202	2024-12-20T23:06:32.439Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5203	2024-12-20T23:06:32.440Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 287d6774-d3e6-4b73-8c77-c90534d4a54d, lookup_type: ById(287d6774-d3e6-4b73-8c77-c90534d4a54d) } result = Ok(())
5204	2024-12-20T23:06:32.440Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5205	2024-12-20T23:06:32.440Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5206	2024-12-20T23:06:32.440Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5207	2024-12-20T23:06:32.441Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 57.083977ms iteration = 26 status = 0
5208	2024-12-20T23:06:32.442Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
5209	2024-12-20T23:06:32.442Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
5210	2024-12-20T23:06:32.442Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5211	2024-12-20T23:06:32.442Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
5212	2024-12-20T23:06:32.446Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
5213	2024-12-20T23:06:32.446Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5214	2024-12-20T23:06:32.447Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "48889574-9e79-4765-a50f-64713af99942", "content-length": "26", "date": "Fri, 20 Dec 2024 23:06:32 GMT"} })
5215	2024-12-20T23:06:32.447Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5216	2024-12-20T23:06:32.448Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
5217	2024-12-20T23:06:32.449Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
5218	2024-12-20T23:06:32.449Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 64.879522ms iteration = 76 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
5219	2024-12-20T23:06:32.451Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 66.509956ms iteration = 28 status = {"physical_disks_added":0}
5220	2024-12-20T23:06:32.469Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 85.006034ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
5221	2024-12-20T23:06:32.472Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
5222	2024-12-20T23:06:32.472Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
5223	2024-12-20T23:06:32.472Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 87.78137ms iteration = 26 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
5224	2024-12-20T23:06:32.498Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5225	2024-12-20T23:06:32.498Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5226	2024-12-20T23:06:32.506Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
5227	2024-12-20T23:06:32.506Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5228	2024-12-20T23:06:32.506Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1f91ae1-cb54-4ec8-b359-c101c7741321", "content-length": "26", "date": "Fri, 20 Dec 2024 23:06:32 GMT"} })
5229	2024-12-20T23:06:32.506Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5230	2024-12-20T23:06:32.507Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5231	2024-12-20T23:06:32.507Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5232	2024-12-20T23:06:32.508Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
5233	2024-12-20T23:06:32.509Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5234	2024-12-20T23:06:32.511Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
5235	2024-12-20T23:06:32.511Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5236	2024-12-20T23:06:32.511Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5237	2024-12-20T23:06:32.512Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5238	2024-12-20T23:06:32.514Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
5239	2024-12-20T23:06:32.514Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5240	2024-12-20T23:06:32.514Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5241	2024-12-20T23:06:32.551Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 zone_address = fd00:1122:3344:101::2
5242	2024-12-20T23:06:32.551Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5243	2024-12-20T23:06:32.552Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "20353f04-ed7e-4ba0-9900-1b224c601f20", "content-length": "26", "date": "Fri, 20 Dec 2024 23:06:32 GMT"} })
5244	2024-12-20T23:06:32.552Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5245	2024-12-20T23:06:32.553Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
5246	2024-12-20T23:06:32.651Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
5247	2024-12-20T23:06:32.652Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "b7288de4-e464-4be3-892e-c68bba4f2242", "content-length": "2", "date": "Fri, 20 Dec 2024 23:06:32 GMT"} })
5248	2024-12-20T23:06:32.653Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5249	2024-12-20T23:06:32.653Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5250	2024-12-20T23:06:32.655Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 271.047512ms iteration = 26 status = null
5251	2024-12-20T23:06:32.656Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
5252	2024-12-20T23:06:32.656Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "111fb482-2688-403d-93c3-d70860161d60", "content-length": "691", "date": "Fri, 20 Dec 2024 23:06:32 GMT"} })
5253	2024-12-20T23:06:32.656Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
5254	2024-12-20T23:06:32.657Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
5255	2024-12-20T23:06:32.657Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
5256	2024-12-20T23:06:32.658Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 273.724556ms iteration = 26 status = {}
5257	2024-12-20T23:06:32.711Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 326.490295ms iteration = 26 status = {}
5258	2024-12-20T23:06:32.712Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
5259	2024-12-20T23:06:32.712Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
5260	2024-12-20T23:06:32.712Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "888460d5-9bba-4514-88e0-9dadea3cb739", "content-length": "372", "date": "Fri, 20 Dec 2024 23:06:32 GMT"} })
5261	2024-12-20T23:06:32.712Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5262	2024-12-20T23:06:32.713Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5263	2024-12-20T23:06:32.769Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
5264	2024-12-20T23:06:32.769Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
5265	2024-12-20T23:06:32.769Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "f0afd876-5772-40c0-9c38-681282e79441", "date": "Fri, 20 Dec 2024 23:06:32 GMT"} })
5266	2024-12-20T23:06:32.769Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5267	2024-12-20T23:06:32.770Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5268	2024-12-20T23:06:32.772Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5269	2024-12-20T23:06:32.772Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5270	2024-12-20T23:06:32.772Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5271	2024-12-20T23:06:32.772Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5272	2024-12-20T23:06:32.774Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 03dd12a0-dc2d-46e1-a7f4-d3397b9b7aa7, rack_id: b394d0c6-19a8-4136-9148-af6c47c1f8e0, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(560ca89c-7c0c-4921-8e5c-b57676f695b0) } rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
5273	2024-12-20T23:06:32.774Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5274	2024-12-20T23:06:32.775Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5275	2024-12-20T23:06:32.775Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5276	2024-12-20T23:06:32.775Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5277	2024-12-20T23:06:32.777Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
5278	2024-12-20T23:06:32.777Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
5279	2024-12-20T23:06:32.778Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "fbd90f03-7f03-4373-a684-3efdf2731fe8", "date": "Fri, 20 Dec 2024 23:06:32 GMT"} })
5280	2024-12-20T23:06:32.780Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 395.206522ms iteration = 26 status = {}
5281	2024-12-20T23:06:32.783Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
5282	2024-12-20T23:06:32.783Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "aafd6bca-a1e2-4706-8611-a2b08662d370", "content-length": "134", "date": "Fri, 20 Dec 2024 23:06:32 GMT"} })
5283	2024-12-20T23:06:32.783Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
5284	2024-12-20T23:06:32.783Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 560ca89c-7c0c-4921-8e5c-b57676f695b0
5285	2024-12-20T23:06:32.783Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
5286	2024-12-20T23:06:32.783Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
5287	2024-12-20T23:06:32.783Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5288	2024-12-20T23:06:32.783Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5289	2024-12-20T23:06:32.784Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5290	2024-12-20T23:06:32.784Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
5291	2024-12-20T23:06:32.784Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5292	2024-12-20T23:06:32.784Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
5293	2024-12-20T23:06:32.784Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
5294	2024-12-20T23:06:32.784Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5295	2024-12-20T23:06:32.784Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5296	2024-12-20T23:06:32.786Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
5297	2024-12-20T23:06:32.786Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "40adffcd-8239-4c64-b6f8-8fff3c4292f3", "content-length": "2", "date": "Fri, 20 Dec 2024 23:06:32 GMT"} })
5298	2024-12-20T23:06:32.786Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
5299	2024-12-20T23:06:32.786Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "44d28be0-11dd-44c8-8357-04944d6b2b13", "content-length": "52", "date": "Fri, 20 Dec 2024 23:06:32 GMT"} })
5300	2024-12-20T23:06:32.786Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
5301	2024-12-20T23:06:32.786Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
5302	2024-12-20T23:06:32.897Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "8a996678-96d2-4b2c-971c-211cf49b3449", "date": "Fri, 20 Dec 2024 23:06:32 GMT"} })
5303	2024-12-20T23:06:32.897Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch = Switch0
5304	2024-12-20T23:06:32.897Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
5305	2024-12-20T23:06:32.897Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "188ba5e8-6bda-4f5d-8933-0149a786df3b", "content-length": "682", "date": "Fri, 20 Dec 2024 23:06:32 GMT"} })
5306	2024-12-20T23:06:32.897Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5307	2024-12-20T23:06:32.897Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5308	2024-12-20T23:06:32.901Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5309	2024-12-20T23:06:32.901Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5310	2024-12-20T23:06:32.905Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5311	2024-12-20T23:06:32.905Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5312	2024-12-20T23:06:32.907Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5313	2024-12-20T23:06:32.908Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5314	2024-12-20T23:06:32.914Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 529.485809ms iteration = 26 status = {}
5315	2024-12-20T23:06:33.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 273 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 83554464-a72a-4706-ba56-a8a464c7ebbd response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
5316	2024-12-20T23:06:42.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 77 reason = Timeout
5317	2024-12-20T23:06:42.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 77 reason = Timeout
5318	2024-12-20T23:06:42.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5319	2024-12-20T23:06:42.385Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5320	2024-12-20T23:06:42.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5321	2024-12-20T23:06:42.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5322	2024-12-20T23:06:42.386Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5323	2024-12-20T23:06:42.387Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5324	2024-12-20T23:06:42.387Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5325	2024-12-20T23:06:42.389Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.025561ms iteration = 77 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
5326	2024-12-20T23:06:42.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5327	2024-12-20T23:06:42.389Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5328	2024-12-20T23:06:42.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5329	2024-12-20T23:06:42.391Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5330	2024-12-20T23:06:42.391Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5331	2024-12-20T23:06:42.393Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5332	2024-12-20T23:06:42.395Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
5333	2024-12-20T23:06:42.395Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5334	2024-12-20T23:06:42.395Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5335	2024-12-20T23:06:42.408Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
5336	2024-12-20T23:06:42.408Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.026082ms iteration = 77 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
5337	2024-12-20T23:06:43.138Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42603 req_id = a9e6382f-3d11-4838-9107-e41a2f39c9f8 roles = RoleSet { roles: {} } uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?sort_by=id_ascending
5338	2024-12-20T23:06:43.139Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42603 req_id = a9e6382f-3d11-4838-9107-e41a2f39c9f8 resource = Database result = Ok(()) uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?sort_by=id_ascending
5339	2024-12-20T23:06:43.141Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2934 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42603 req_id = a9e6382f-3d11-4838-9107-e41a2f39c9f8 response_code = 200 uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?sort_by=id_ascending
5340	2024-12-20T23:06:43.141Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42603 req_id = 8306c75b-0743-474e-837f-251a00f12a93 roles = RoleSet { roles: {} } uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmM2I2MGJkMy02ZTM0LTQxNDAtYTc1NS01N2YyNmRiYzViMzQifX0%3D
5341	2024-12-20T23:06:43.142Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42603 req_id = 8306c75b-0743-474e-837f-251a00f12a93 resource = Database result = Ok(()) uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmM2I2MGJkMy02ZTM0LTQxNDAtYTc1NS01N2YyNmRiYzViMzQifX0%3D
5342	2024-12-20T23:06:43.144Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2537 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42603 req_id = 8306c75b-0743-474e-837f-251a00f12a93 response_code = 200 uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmM2I2MGJkMy02ZTM0LTQxNDAtYTc1NS01N2YyNmRiYzViMzQifX0%3D
5343	2024-12-20T23:06:43.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 254 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = aae6f395-4f02-4387-a86d-96d6906a3326 response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
5344	2024-12-20T23:06:52.384Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 78 reason = Timeout
5345	2024-12-20T23:06:52.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 78 reason = Timeout
5346	2024-12-20T23:06:52.384Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5347	2024-12-20T23:06:52.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5348	2024-12-20T23:06:52.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5349	2024-12-20T23:06:52.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5350	2024-12-20T23:06:52.387Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5351	2024-12-20T23:06:52.387Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5352	2024-12-20T23:06:52.387Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5353	2024-12-20T23:06:52.389Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.341099ms iteration = 78 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
5354	2024-12-20T23:06:52.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5355	2024-12-20T23:06:52.390Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5356	2024-12-20T23:06:52.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5357	2024-12-20T23:06:52.391Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5358	2024-12-20T23:06:52.391Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5359	2024-12-20T23:06:52.393Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5360	2024-12-20T23:06:52.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
5361	2024-12-20T23:06:52.396Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5362	2024-12-20T23:06:52.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5363	2024-12-20T23:06:52.405Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
5364	2024-12-20T23:06:52.405Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.547105ms iteration = 78 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
5365	2024-12-20T23:06:53.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 351 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 61adac99-b5f2-4bc1-bc65-a274b7f67b7a response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
5366	2024-12-20T23:07:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 27 reason = Timeout
5367	2024-12-20T23:07:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
5368	2024-12-20T23:07:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 15 reason = Timeout
5369	2024-12-20T23:07:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 27 reason = Timeout
5370	2024-12-20T23:07:02.384Z	WARN	nexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7
5371	2024-12-20T23:07:02.384Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
5372	2024-12-20T23:07:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5373	2024-12-20T23:07:02.384Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 23.191\u{b5}s iteration = 15 status = {"enabled":false,"target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7"}
5374	2024-12-20T23:07:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 27 reason = Timeout
5375	2024-12-20T23:07:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 27 reason = Timeout
5376	2024-12-20T23:07:02.384Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
5377	2024-12-20T23:07:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5378	2024-12-20T23:07:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5379	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 14 reason = Timeout
5380	2024-12-20T23:07:02.385Z	INFO	nexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
5381	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
5382	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 27 reason = Timeout
5383	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 27 reason = Timeout
5384	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 14 reason = Timeout
5385	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5386	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
5387	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 14 reason = Timeout
5388	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 14 reason = Timeout
5389	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
5390	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 27 reason = Timeout
5391	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5392	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 14 reason = Timeout
5393	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
5394	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5395	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 27 reason = Timeout
5396	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 27 reason = Timeout
5397	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5398	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
5399	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 16 reason = Timeout
5400	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 14 reason = Timeout
5401	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
5402	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 29 reason = Timeout
5403	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5404	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 27 reason = Timeout
5405	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
5406	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 27 reason = Timeout
5407	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5408	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 14 reason = Timeout
5409	2024-12-20T23:07:02.385Z	INFO	nexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-20 22:57:02.385200164 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
5410	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 79 reason = Timeout
5411	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
5412	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 79 reason = Timeout
5413	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5414	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5415	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 27 reason = Timeout
5416	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 14 reason = Timeout
5417	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 14 reason = Timeout
5418	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = external_endpoints iteration = 14 reason = Timeout
5419	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
5420	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5421	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5422	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5423	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 27 reason = Timeout
5424	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
5425	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 16 reason = Timeout
5426	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 27 reason = Timeout
5427	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
5428	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
5429	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
5430	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
5431	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5432	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5433	2024-12-20T23:07:02.385Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("616ce87c-1f39-49d9-ba35-ee63b2650ecd.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b4615881-2f68-4c82-be17-071bf3ae3641.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c9217e92-0af3-4930-acb2-aa82ded5b782.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("c9217e92-0af3-4930-acb2-aa82ded5b782.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 1437, tv_nsec: 775407176 } })
5434	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
5435	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5436	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
5437	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
5438	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
5439	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5440	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
5441	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5442	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5443	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5444	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5445	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
5446	2024-12-20T23:07:02.385Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1e296b8d-6800-4056-8034-a1ec2d005fff.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("ace0592c-7755-46f8-b47a-c9b6444fe64c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("ace0592c-7755-46f8-b47a-c9b6444fe64c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1437, tv_nsec: 775638913 } })
5447	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5448	2024-12-20T23:07:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5449	2024-12-20T23:07:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
5450	2024-12-20T23:07:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5451	2024-12-20T23:07:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5452	2024-12-20T23:07:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
5453	2024-12-20T23:07:02.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
5454	2024-12-20T23:07:02.386Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
5455	2024-12-20T23:07:02.386Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.528532ms iteration = 14 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
5456	2024-12-20T23:07:02.386Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
5457	2024-12-20T23:07:02.386Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.264635ms iteration = 14 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
5458	2024-12-20T23:07:02.390Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5459	2024-12-20T23:07:02.391Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5460	2024-12-20T23:07:02.391Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5461	2024-12-20T23:07:02.391Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5462	2024-12-20T23:07:02.391Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5463	2024-12-20T23:07:02.392Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 7.353682ms iteration = 27 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
5464	2024-12-20T23:07:02.392Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1437, tv_nsec: 782393858 } })
5465	2024-12-20T23:07:02.392Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
5466	2024-12-20T23:07:02.393Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
5467	2024-12-20T23:07:02.393Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5468	2024-12-20T23:07:02.393Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
5469	2024-12-20T23:07:02.393Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
5470	2024-12-20T23:07:02.393Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5471	2024-12-20T23:07:02.394Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
5472	2024-12-20T23:07:02.394Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
5473	2024-12-20T23:07:02.394Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5474	2024-12-20T23:07:02.394Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5475	2024-12-20T23:07:02.396Z	DEBG	nexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 11.158067ms iteration = 14 status = {"errors":[],"found_port_ok":[]}
5476	2024-12-20T23:07:02.397Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 12.37716ms iteration = 27 status = {"errors":[],"finish_invoked_ok":[]}
5477	2024-12-20T23:07:02.397Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5478	2024-12-20T23:07:02.398Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5479	2024-12-20T23:07:02.398Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5480	2024-12-20T23:07:02.399Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
5481	2024-12-20T23:07:02.399Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5482	2024-12-20T23:07:02.399Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5483	2024-12-20T23:07:02.401Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5484	2024-12-20T23:07:02.401Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1437, tv_nsec: 791654802 } })
5485	2024-12-20T23:07:02.402Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 16.665247ms iteration = 27 status = {"errors":[],"garbage_collect_requested":[]}
5486	2024-12-20T23:07:02.402Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
5487	2024-12-20T23:07:02.402Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
5488	2024-12-20T23:07:02.402Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5489	2024-12-20T23:07:02.403Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5490	2024-12-20T23:07:02.403Z	INFO	nexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-20 22:57:02.385200164 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
5491	2024-12-20T23:07:02.403Z	DEBG	nexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 18.022195ms iteration = 14 status = {"expiration":"2024-12-20T22:57:02.385200164Z","pruned":[]}
5492	2024-12-20T23:07:02.403Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5493	2024-12-20T23:07:02.403Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5494	2024-12-20T23:07:02.404Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5495	2024-12-20T23:07:02.405Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5496	2024-12-20T23:07:02.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5497	2024-12-20T23:07:02.407Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5498	2024-12-20T23:07:02.407Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5499	2024-12-20T23:07:02.408Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
5500	2024-12-20T23:07:02.408Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5501	2024-12-20T23:07:02.409Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5502	2024-12-20T23:07:02.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5503	2024-12-20T23:07:02.410Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5504	2024-12-20T23:07:02.410Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 25.068448ms iteration = 79 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
5505	2024-12-20T23:07:02.410Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5506	2024-12-20T23:07:02.410Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5507	2024-12-20T23:07:02.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5508	2024-12-20T23:07:02.411Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5509	2024-12-20T23:07:02.411Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
5510	2024-12-20T23:07:02.411Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5511	2024-12-20T23:07:02.411Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5512	2024-12-20T23:07:02.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5513	2024-12-20T23:07:02.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5514	2024-12-20T23:07:02.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5515	2024-12-20T23:07:02.413Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5516	2024-12-20T23:07:02.414Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5517	2024-12-20T23:07:02.414Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5518	2024-12-20T23:07:02.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
5519	2024-12-20T23:07:02.415Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5520	2024-12-20T23:07:02.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5521	2024-12-20T23:07:02.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
5522	2024-12-20T23:07:02.418Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5523	2024-12-20T23:07:02.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
5524	2024-12-20T23:07:02.418Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5525	2024-12-20T23:07:02.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5526	2024-12-20T23:07:02.418Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5527	2024-12-20T23:07:02.420Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
5528	2024-12-20T23:07:02.420Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
5529	2024-12-20T23:07:02.429Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5530	2024-12-20T23:07:02.430Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
5531	2024-12-20T23:07:02.430Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
5532	2024-12-20T23:07:02.430Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5533	2024-12-20T23:07:02.431Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
5534	2024-12-20T23:07:02.431Z	DEBG	nexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
5535	2024-12-20T23:07:02.431Z	INFO	nexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
5536	2024-12-20T23:07:02.431Z	DEBG	nexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 46.308962ms iteration = 14 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
5537	2024-12-20T23:07:02.431Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
5538	2024-12-20T23:07:02.431Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5539	2024-12-20T23:07:02.431Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
5540	2024-12-20T23:07:02.432Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5541	2024-12-20T23:07:02.432Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5542	2024-12-20T23:07:02.437Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5543	2024-12-20T23:07:02.438Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5544	2024-12-20T23:07:02.442Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5545	2024-12-20T23:07:02.443Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
5546	2024-12-20T23:07:02.444Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
5547	2024-12-20T23:07:02.444Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
5548	2024-12-20T23:07:02.444Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5549	2024-12-20T23:07:02.444Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5550	2024-12-20T23:07:02.447Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5551	2024-12-20T23:07:02.447Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
5552	2024-12-20T23:07:02.447Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
5553	2024-12-20T23:07:02.447Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
5554	2024-12-20T23:07:02.447Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5555	2024-12-20T23:07:02.447Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5556	2024-12-20T23:07:02.448Z	DEBG	nexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
5557	2024-12-20T23:07:02.448Z	INFO	nexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
5558	2024-12-20T23:07:02.448Z	DEBG	nexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 63.699329ms iteration = 14 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
5559	2024-12-20T23:07:02.449Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
5560	2024-12-20T23:07:02.450Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
5561	2024-12-20T23:07:02.450Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
5562	2024-12-20T23:07:02.450Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
5563	2024-12-20T23:07:02.450Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5564	2024-12-20T23:07:02.451Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5565	2024-12-20T23:07:02.452Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5566	2024-12-20T23:07:02.452Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5567	2024-12-20T23:07:02.458Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
5568	2024-12-20T23:07:02.458Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
5569	2024-12-20T23:07:02.511Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
5570	2024-12-20T23:07:02.524Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
5571	2024-12-20T23:07:02.524Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5572	2024-12-20T23:07:02.524Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "caa7c7e4-074f-4eaa-bafa-4d7a48510064", "content-length": "26", "date": "Fri, 20 Dec 2024 23:07:02 GMT"} })
5573	2024-12-20T23:07:02.524Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5574	2024-12-20T23:07:02.527Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
5575	2024-12-20T23:07:02.559Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
5576	2024-12-20T23:07:02.559Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5577	2024-12-20T23:07:02.559Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "70661b29-b943-48ff-8066-bcdc7cae9924", "content-length": "26", "date": "Fri, 20 Dec 2024 23:07:02 GMT"} })
5578	2024-12-20T23:07:02.559Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5579	2024-12-20T23:07:02.561Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
5580	2024-12-20T23:07:02.592Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
5581	2024-12-20T23:07:02.652Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
5582	2024-12-20T23:07:02.688Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
5583	2024-12-20T23:07:02.688Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
5584	2024-12-20T23:07:02.688Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "9daede92-b618-4c29-889f-e52f5b119813", "content-length": "372", "date": "Fri, 20 Dec 2024 23:07:02 GMT"} })
5585	2024-12-20T23:07:02.688Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5586	2024-12-20T23:07:02.689Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5587	2024-12-20T23:07:02.693Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
5588	2024-12-20T23:07:02.693Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
5589	2024-12-20T23:07:02.693Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5590	2024-12-20T23:07:02.693Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
5591	2024-12-20T23:07:02.694Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5592	2024-12-20T23:07:02.694Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5593	2024-12-20T23:07:02.694Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5594	2024-12-20T23:07:02.695Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1438, tv_nsec: 84846873 } })
5595	2024-12-20T23:07:02.695Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
5596	2024-12-20T23:07:02.695Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 310.760163ms iteration = 29 status = {"physical_disks_added":0}
5597	2024-12-20T23:07:02.698Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
5598	2024-12-20T23:07:02.698Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5599	2024-12-20T23:07:02.698Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5600	2024-12-20T23:07:02.699Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 314.853156ms iteration = 27 status = {}
5601	2024-12-20T23:07:02.701Z	WARN	nexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
5602	2024-12-20T23:07:02.701Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 316.223532ms iteration = 14 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
5603	2024-12-20T23:07:02.712Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
5604	2024-12-20T23:07:02.712Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 326.944247ms iteration = 79 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
5605	2024-12-20T23:07:02.712Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
5606	2024-12-20T23:07:02.712Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "9dbdd995-c5be-4bdd-955b-778f7c8aa09d", "content-length": "2", "date": "Fri, 20 Dec 2024 23:07:02 GMT"} })
5607	2024-12-20T23:07:02.714Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5608	2024-12-20T23:07:02.714Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5609	2024-12-20T23:07:02.716Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 331.455031ms iteration = 27 status = null
5610	2024-12-20T23:07:02.727Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5611	2024-12-20T23:07:02.728Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5612	2024-12-20T23:07:02.733Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5613	2024-12-20T23:07:02.734Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 348.758326ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
5614	2024-12-20T23:07:02.736Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5615	2024-12-20T23:07:02.736Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5616	2024-12-20T23:07:02.736Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 351.758689ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
5617	2024-12-20T23:07:02.738Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
5618	2024-12-20T23:07:02.738Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5619	2024-12-20T23:07:02.738Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5620	2024-12-20T23:07:02.739Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5621	2024-12-20T23:07:02.739Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
5622	2024-12-20T23:07:02.739Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 354.845704ms iteration = 27 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
5623	2024-12-20T23:07:02.740Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5624	2024-12-20T23:07:02.741Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
5625	2024-12-20T23:07:02.741Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 5fa20c62-75c0-4531-be07-51b87d303d18 dns_zone_name = oxide.test found_so_far = 0 version = 2
5626	2024-12-20T23:07:02.741Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5627	2024-12-20T23:07:02.741Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5628	2024-12-20T23:07:02.742Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
5629	2024-12-20T23:07:02.742Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5630	2024-12-20T23:07:02.742Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5631	2024-12-20T23:07:02.742Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
5632	2024-12-20T23:07:02.742Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5633	2024-12-20T23:07:02.743Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5634	2024-12-20T23:07:02.743Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5635	2024-12-20T23:07:02.744Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5636	2024-12-20T23:07:02.746Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
5637	2024-12-20T23:07:02.746Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
5638	2024-12-20T23:07:02.746Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 361.308231ms iteration = 27 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
5639	2024-12-20T23:07:02.746Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
5640	2024-12-20T23:07:02.746Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5641	2024-12-20T23:07:02.747Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5642	2024-12-20T23:07:02.748Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
5643	2024-12-20T23:07:02.748Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 1e86bf23-6ffd-4a12-93c7-0061d7d17590 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
5644	2024-12-20T23:07:02.748Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5645	2024-12-20T23:07:02.748Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
5646	2024-12-20T23:07:02.748Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5647	2024-12-20T23:07:02.748Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
5648	2024-12-20T23:07:02.748Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5649	2024-12-20T23:07:02.748Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5650	2024-12-20T23:07:02.749Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "b3e42fc9-6c5a-4736-834e-4008137887bc", "date": "Fri, 20 Dec 2024 23:07:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
5651	2024-12-20T23:07:02.749Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
5652	2024-12-20T23:07:02.750Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5653	2024-12-20T23:07:02.753Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "a910f1eb-7027-4f10-a918-9d7a5e79603b", "date": "Fri, 20 Dec 2024 23:07:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
5654	2024-12-20T23:07:02.753Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
5655	2024-12-20T23:07:02.754Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 287d6774-d3e6-4b73-8c77-c90534d4a54d, lookup_type: ById(287d6774-d3e6-4b73-8c77-c90534d4a54d) } result = Ok(())
5656	2024-12-20T23:07:02.754Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5657	2024-12-20T23:07:02.754Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5658	2024-12-20T23:07:02.757Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 5fa20c62-75c0-4531-be07-51b87d303d18 dns_zone_name = oxide.test found_so_far = 1 version = 2
5659	2024-12-20T23:07:02.757Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
5660	2024-12-20T23:07:02.757Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-20 22:54:39.899407 UTC dns_group = external generation = 2
5661	2024-12-20T23:07:02.757Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 372.162699ms iteration = 14 status = {"generation":2}
5662	2024-12-20T23:07:02.763Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
5663	2024-12-20T23:07:02.763Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 378.595975ms iteration = 16 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
5664	2024-12-20T23:07:02.763Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 378.938465ms iteration = 27 status = 0
5665	2024-12-20T23:07:02.787Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
5666	2024-12-20T23:07:02.787Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
5667	2024-12-20T23:07:02.788Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "fcbbf08e-3f8b-4259-ae53-efc7c100b8c7", "date": "Fri, 20 Dec 2024 23:07:02 GMT"} })
5668	2024-12-20T23:07:02.788Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5669	2024-12-20T23:07:02.788Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5670	2024-12-20T23:07:02.788Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
5671	2024-12-20T23:07:02.789Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "d8c28fae-0b82-435c-897c-7efa82ba81ef", "date": "Fri, 20 Dec 2024 23:07:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
5672	2024-12-20T23:07:02.789Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
5673	2024-12-20T23:07:02.789Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 404.59481ms iteration = 27 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
5674	2024-12-20T23:07:02.791Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "c60bdfa2-26f3-4eef-b253-1406237adeb8", "date": "Fri, 20 Dec 2024 23:07:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
5675	2024-12-20T23:07:02.791Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
5676	2024-12-20T23:07:02.791Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5677	2024-12-20T23:07:02.791Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 zone_address = fd00:1122:3344:101::2
5678	2024-12-20T23:07:02.791Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5679	2024-12-20T23:07:02.791Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5680	2024-12-20T23:07:02.791Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "68161199-7f88-42a3-a34d-07babe690e4c", "content-length": "26", "date": "Fri, 20 Dec 2024 23:07:02 GMT"} })
5681	2024-12-20T23:07:02.791Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5682	2024-12-20T23:07:02.793Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "f049941a-b129-460e-b0db-1e05defd0616", "date": "Fri, 20 Dec 2024 23:07:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
5683	2024-12-20T23:07:02.793Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
5684	2024-12-20T23:07:02.794Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
5685	2024-12-20T23:07:02.795Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5686	2024-12-20T23:07:02.795Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5687	2024-12-20T23:07:02.796Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
5688	2024-12-20T23:07:02.796Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 410.703577ms iteration = 16 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
5689	2024-12-20T23:07:02.799Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
5690	2024-12-20T23:07:02.799Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
5691	2024-12-20T23:07:02.799Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "35e08ae6-bbe2-43f0-a367-7fcb8a1a5fc8", "date": "Fri, 20 Dec 2024 23:07:02 GMT"} })
5692	2024-12-20T23:07:02.800Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 416.078375ms iteration = 27 status = {}
5693	2024-12-20T23:07:02.804Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5694	2024-12-20T23:07:02.807Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
5695	2024-12-20T23:07:02.807Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5696	2024-12-20T23:07:02.807Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5697	2024-12-20T23:07:02.809Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 1e86bf23-6ffd-4a12-93c7-0061d7d17590 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
5698	2024-12-20T23:07:02.809Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
5699	2024-12-20T23:07:02.809Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-20 22:54:38.956071 UTC dns_group = internal generation = 1
5700	2024-12-20T23:07:02.809Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 424.421874ms iteration = 14 status = {"generation":1}
5701	2024-12-20T23:07:02.811Z	DEBG	nexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-20T22:53:58.073373Z, time_modified: 2024-12-20T22:53:58.073373Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, db_silo: Silo { identity: SiloIdentity { id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-20T22:54:39.847015Z, time_modified: 2024-12-20T22:54:39.847015Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, db_silo: Silo { identity: SiloIdentity { id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-20T22:54:39.847015Z, time_modified: 2024-12-20T22:54:39.847015Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156" }] }) }
5702	2024-12-20T23:07:02.811Z	DEBG	nexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 426.47464ms iteration = 14 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"ee579470-b16b-4af1-ad3a-b3d29f9a2fb2","tls_certs":["ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156"]}},"default_endpoint":{"silo_id":"ee579470-b16b-4af1-ad3a-b3d29f9a2fb2","tls_certs":["ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
5703	2024-12-20T23:07:02.893Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
5704	2024-12-20T23:07:02.893Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "5eb0a6d1-3b86-4105-b729-e800291a98b6", "content-length": "691", "date": "Fri, 20 Dec 2024 23:07:02 GMT"} })
5705	2024-12-20T23:07:02.893Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
5706	2024-12-20T23:07:02.893Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
5707	2024-12-20T23:07:02.894Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
5708	2024-12-20T23:07:02.896Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 511.257348ms iteration = 27 status = {}
5709	2024-12-20T23:07:02.915Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5710	2024-12-20T23:07:02.915Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5711	2024-12-20T23:07:02.917Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 03dd12a0-dc2d-46e1-a7f4-d3397b9b7aa7, rack_id: b394d0c6-19a8-4136-9148-af6c47c1f8e0, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(560ca89c-7c0c-4921-8e5c-b57676f695b0) } rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
5712	2024-12-20T23:07:02.917Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5713	2024-12-20T23:07:02.917Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5714	2024-12-20T23:07:02.927Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
5715	2024-12-20T23:07:02.927Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a35054c-cc7f-47da-9224-d6fb0708c34b", "content-length": "134", "date": "Fri, 20 Dec 2024 23:07:02 GMT"} })
5716	2024-12-20T23:07:02.927Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
5717	2024-12-20T23:07:02.927Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 560ca89c-7c0c-4921-8e5c-b57676f695b0
5718	2024-12-20T23:07:02.928Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
5719	2024-12-20T23:07:02.928Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
5720	2024-12-20T23:07:02.928Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5721	2024-12-20T23:07:02.928Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5722	2024-12-20T23:07:02.928Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5723	2024-12-20T23:07:02.928Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
5724	2024-12-20T23:07:02.928Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5725	2024-12-20T23:07:02.928Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
5726	2024-12-20T23:07:02.928Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
5727	2024-12-20T23:07:02.928Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5728	2024-12-20T23:07:02.928Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5729	2024-12-20T23:07:02.930Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
5730	2024-12-20T23:07:02.930Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b7fc08b-6af6-4895-ac4d-070d7f33fbe1", "content-length": "2", "date": "Fri, 20 Dec 2024 23:07:02 GMT"} })
5731	2024-12-20T23:07:02.930Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
5732	2024-12-20T23:07:02.931Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "06e532da-dc59-4533-ab9c-4865951a0f57", "content-length": "52", "date": "Fri, 20 Dec 2024 23:07:02 GMT"} })
5733	2024-12-20T23:07:02.931Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
5734	2024-12-20T23:07:02.931Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
5735	2024-12-20T23:07:03.032Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "7c4c985a-744a-4e3d-af01-884ce2a2ef61", "date": "Fri, 20 Dec 2024 23:07:02 GMT"} })
5736	2024-12-20T23:07:03.032Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch = Switch0
5737	2024-12-20T23:07:03.032Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
5738	2024-12-20T23:07:03.040Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "31b2120b-2861-4c3d-b794-706becbf801b", "content-length": "682", "date": "Fri, 20 Dec 2024 23:07:02 GMT"} })
5739	2024-12-20T23:07:03.040Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5740	2024-12-20T23:07:03.040Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5741	2024-12-20T23:07:03.044Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5742	2024-12-20T23:07:03.044Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5743	2024-12-20T23:07:03.048Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5744	2024-12-20T23:07:03.048Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5745	2024-12-20T23:07:03.050Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5746	2024-12-20T23:07:03.050Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5747	2024-12-20T23:07:03.058Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 673.104412ms iteration = 27 status = {}
5748	2024-12-20T23:07:03.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 336 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = db61203e-cbac-4b43-9499-d9ec8d2b239c response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
5749	2024-12-20T23:07:12.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 80 reason = Timeout
5750	2024-12-20T23:07:12.385Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5751	2024-12-20T23:07:12.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 80 reason = Timeout
5752	2024-12-20T23:07:12.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5753	2024-12-20T23:07:12.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5754	2024-12-20T23:07:12.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5755	2024-12-20T23:07:12.387Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5756	2024-12-20T23:07:12.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5757	2024-12-20T23:07:12.389Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5758	2024-12-20T23:07:12.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5759	2024-12-20T23:07:12.392Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5760	2024-12-20T23:07:12.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5761	2024-12-20T23:07:12.394Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5762	2024-12-20T23:07:12.397Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
5763	2024-12-20T23:07:12.397Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5764	2024-12-20T23:07:12.397Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5765	2024-12-20T23:07:12.409Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
5766	2024-12-20T23:07:12.409Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.755099ms iteration = 80 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
5767	2024-12-20T23:07:12.413Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5768	2024-12-20T23:07:12.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5769	2024-12-20T23:07:12.442Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 56.84823ms iteration = 80 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
5770	2024-12-20T23:07:13.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 287 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 20ed0079-79a2-4600-998a-89dcf3f196cd response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
5771	2024-12-20T23:07:22.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 81 reason = Timeout
5772	2024-12-20T23:07:22.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5773	2024-12-20T23:07:22.384Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 81 reason = Timeout
5774	2024-12-20T23:07:22.384Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5775	2024-12-20T23:07:22.384Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5776	2024-12-20T23:07:22.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5777	2024-12-20T23:07:22.386Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5778	2024-12-20T23:07:22.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5779	2024-12-20T23:07:22.389Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5780	2024-12-20T23:07:22.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5781	2024-12-20T23:07:22.390Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5782	2024-12-20T23:07:22.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5783	2024-12-20T23:07:22.392Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5784	2024-12-20T23:07:22.394Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
5785	2024-12-20T23:07:22.394Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5786	2024-12-20T23:07:22.394Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5787	2024-12-20T23:07:22.404Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
5788	2024-12-20T23:07:22.404Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.02511ms iteration = 81 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
5789	2024-12-20T23:07:22.415Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5790	2024-12-20T23:07:22.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5791	2024-12-20T23:07:22.417Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.581455ms iteration = 81 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
5792	2024-12-20T23:07:23.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 310 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 4fffac2d-d5fc-428f-9b74-2a6618fb75d9 response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
5793	2024-12-20T23:07:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 28 reason = Timeout
5794	2024-12-20T23:07:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
5795	2024-12-20T23:07:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 28 reason = Timeout
5796	2024-12-20T23:07:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5797	2024-12-20T23:07:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 30 reason = Timeout
5798	2024-12-20T23:07:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 82 reason = Timeout
5799	2024-12-20T23:07:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5800	2024-12-20T23:07:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5801	2024-12-20T23:07:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 28 reason = Timeout
5802	2024-12-20T23:07:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
5803	2024-12-20T23:07:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 82 reason = Timeout
5804	2024-12-20T23:07:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5805	2024-12-20T23:07:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 28 reason = Timeout
5806	2024-12-20T23:07:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5807	2024-12-20T23:07:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 28 reason = Timeout
5808	2024-12-20T23:07:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5809	2024-12-20T23:07:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 28 reason = Timeout
5810	2024-12-20T23:07:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 28 reason = Timeout
5811	2024-12-20T23:07:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5812	2024-12-20T23:07:32.384Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
5813	2024-12-20T23:07:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5814	2024-12-20T23:07:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 28 reason = Timeout
5815	2024-12-20T23:07:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
5816	2024-12-20T23:07:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 28 reason = Timeout
5817	2024-12-20T23:07:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
5818	2024-12-20T23:07:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 28 reason = Timeout
5819	2024-12-20T23:07:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
5820	2024-12-20T23:07:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 28 reason = Timeout
5821	2024-12-20T23:07:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5822	2024-12-20T23:07:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 28 reason = Timeout
5823	2024-12-20T23:07:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 28 reason = Timeout
5824	2024-12-20T23:07:32.384Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
5825	2024-12-20T23:07:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5826	2024-12-20T23:07:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 28 reason = Timeout
5827	2024-12-20T23:07:32.384Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5828	2024-12-20T23:07:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
5829	2024-12-20T23:07:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5830	2024-12-20T23:07:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5831	2024-12-20T23:07:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
5832	2024-12-20T23:07:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5833	2024-12-20T23:07:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5834	2024-12-20T23:07:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5835	2024-12-20T23:07:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5836	2024-12-20T23:07:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
5837	2024-12-20T23:07:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
5838	2024-12-20T23:07:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
5839	2024-12-20T23:07:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5840	2024-12-20T23:07:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
5841	2024-12-20T23:07:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
5842	2024-12-20T23:07:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5843	2024-12-20T23:07:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5844	2024-12-20T23:07:32.387Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1467, tv_nsec: 778088200 } })
5845	2024-12-20T23:07:32.388Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
5846	2024-12-20T23:07:32.388Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5847	2024-12-20T23:07:32.388Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.811745ms iteration = 28 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
5848	2024-12-20T23:07:32.388Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5849	2024-12-20T23:07:32.388Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5850	2024-12-20T23:07:32.390Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5851	2024-12-20T23:07:32.391Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5852	2024-12-20T23:07:32.391Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5853	2024-12-20T23:07:32.391Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5854	2024-12-20T23:07:32.392Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
5855	2024-12-20T23:07:32.392Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5856	2024-12-20T23:07:32.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5857	2024-12-20T23:07:32.394Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5858	2024-12-20T23:07:32.397Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
5859	2024-12-20T23:07:32.397Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5860	2024-12-20T23:07:32.397Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
5861	2024-12-20T23:07:32.401Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1467, tv_nsec: 791506708 } })
5862	2024-12-20T23:07:32.401Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 16.591185ms iteration = 82 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
5863	2024-12-20T23:07:32.401Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
5864	2024-12-20T23:07:32.402Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5865	2024-12-20T23:07:32.402Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5866	2024-12-20T23:07:32.421Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5867	2024-12-20T23:07:32.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5868	2024-12-20T23:07:32.424Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5869	2024-12-20T23:07:32.426Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
5870	2024-12-20T23:07:32.426Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5871	2024-12-20T23:07:32.426Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5872	2024-12-20T23:07:32.429Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5873	2024-12-20T23:07:32.429Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5874	2024-12-20T23:07:32.431Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5875	2024-12-20T23:07:32.432Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
5876	2024-12-20T23:07:32.432Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
5877	2024-12-20T23:07:32.433Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 48.308716ms iteration = 28 status = {"errors":[],"finish_invoked_ok":[]}
5878	2024-12-20T23:07:32.433Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
5879	2024-12-20T23:07:32.433Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5880	2024-12-20T23:07:32.434Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ecfae8af-4e75-44a3-b5c7-e8c93effab25", "content-length": "26", "date": "Fri, 20 Dec 2024 23:07:32 GMT"} })
5881	2024-12-20T23:07:32.434Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5882	2024-12-20T23:07:32.435Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
5883	2024-12-20T23:07:32.435Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5884	2024-12-20T23:07:32.435Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5885	2024-12-20T23:07:32.435Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 51.00506ms iteration = 28 status = {"errors":[],"garbage_collect_requested":[]}
5886	2024-12-20T23:07:32.435Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
5887	2024-12-20T23:07:32.437Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5888	2024-12-20T23:07:32.437Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5889	2024-12-20T23:07:32.437Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5890	2024-12-20T23:07:32.438Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
5891	2024-12-20T23:07:32.439Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
5892	2024-12-20T23:07:32.439Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5893	2024-12-20T23:07:32.440Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
5894	2024-12-20T23:07:32.440Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
5895	2024-12-20T23:07:32.440Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5896	2024-12-20T23:07:32.440Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
5897	2024-12-20T23:07:32.440Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5898	2024-12-20T23:07:32.440Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5899	2024-12-20T23:07:32.440Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 55.889835ms iteration = 28 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
5900	2024-12-20T23:07:32.440Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
5901	2024-12-20T23:07:32.441Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
5902	2024-12-20T23:07:32.441Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5903	2024-12-20T23:07:32.441Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
5904	2024-12-20T23:07:32.441Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5905	2024-12-20T23:07:32.441Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5906	2024-12-20T23:07:32.441Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 57.357945ms iteration = 30 status = {"physical_disks_added":0}
5907	2024-12-20T23:07:32.442Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5908	2024-12-20T23:07:32.442Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
5909	2024-12-20T23:07:32.442Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5910	2024-12-20T23:07:32.442Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5911	2024-12-20T23:07:32.444Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
5912	2024-12-20T23:07:32.444Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5913	2024-12-20T23:07:32.444Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e93b424f-4dc2-4878-85c9-561305265b47", "content-length": "26", "date": "Fri, 20 Dec 2024 23:07:32 GMT"} })
5914	2024-12-20T23:07:32.444Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5915	2024-12-20T23:07:32.445Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5916	2024-12-20T23:07:32.445Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5917	2024-12-20T23:07:32.446Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
5918	2024-12-20T23:07:32.446Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1467, tv_nsec: 836909345 } })
5919	2024-12-20T23:07:32.446Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
5920	2024-12-20T23:07:32.448Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
5921	2024-12-20T23:07:32.448Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 63.39318ms iteration = 28 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
5922	2024-12-20T23:07:32.458Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
5923	2024-12-20T23:07:32.458Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 73.842457ms iteration = 82 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
5924	2024-12-20T23:07:32.459Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5925	2024-12-20T23:07:32.460Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5926	2024-12-20T23:07:32.467Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 83.130502ms iteration = 28 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
5927	2024-12-20T23:07:32.468Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5928	2024-12-20T23:07:32.469Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5929	2024-12-20T23:07:32.470Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5930	2024-12-20T23:07:32.471Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5931	2024-12-20T23:07:32.472Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5932	2024-12-20T23:07:32.475Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 287d6774-d3e6-4b73-8c77-c90534d4a54d, lookup_type: ById(287d6774-d3e6-4b73-8c77-c90534d4a54d) } result = Ok(())
5933	2024-12-20T23:07:32.475Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5934	2024-12-20T23:07:32.475Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5935	2024-12-20T23:07:32.476Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
5936	2024-12-20T23:07:32.476Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
5937	2024-12-20T23:07:32.476Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5938	2024-12-20T23:07:32.476Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
5939	2024-12-20T23:07:32.476Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 92.182202ms iteration = 28 status = 0
5940	2024-12-20T23:07:32.491Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 106.832854ms iteration = 28 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
5941	2024-12-20T23:07:32.501Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5942	2024-12-20T23:07:32.501Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5943	2024-12-20T23:07:32.531Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 146.475932ms iteration = 28 status = {}
5944	2024-12-20T23:07:32.550Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
5945	2024-12-20T23:07:32.550Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
5946	2024-12-20T23:07:32.551Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "9532f2ec-c269-402a-a7c5-b1010f51cdab", "content-length": "372", "date": "Fri, 20 Dec 2024 23:07:32 GMT"} })
5947	2024-12-20T23:07:32.551Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5948	2024-12-20T23:07:32.551Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5949	2024-12-20T23:07:32.586Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
5950	2024-12-20T23:07:32.586Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
5951	2024-12-20T23:07:32.586Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 202.049968ms iteration = 28 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
5952	2024-12-20T23:07:32.596Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
5953	2024-12-20T23:07:32.596Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
5954	2024-12-20T23:07:32.596Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "5d6659db-161f-4121-abac-3eba2853caef", "date": "Fri, 20 Dec 2024 23:07:32 GMT"} })
5955	2024-12-20T23:07:32.596Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5956	2024-12-20T23:07:32.596Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5957	2024-12-20T23:07:32.599Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5958	2024-12-20T23:07:32.599Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5959	2024-12-20T23:07:32.601Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5960	2024-12-20T23:07:32.602Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5961	2024-12-20T23:07:32.605Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
5962	2024-12-20T23:07:32.605Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
5963	2024-12-20T23:07:32.605Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "ead361f1-6185-4bf2-a66b-5d8448678e0b", "date": "Fri, 20 Dec 2024 23:07:32 GMT"} })
5964	2024-12-20T23:07:32.606Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 222.051087ms iteration = 28 status = {}
5965	2024-12-20T23:07:32.617Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 zone_address = fd00:1122:3344:101::2
5966	2024-12-20T23:07:32.617Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5967	2024-12-20T23:07:32.618Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
5968	2024-12-20T23:07:32.618Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "acacdc3f-ea08-451a-828a-48b13b8da8c8", "content-length": "26", "date": "Fri, 20 Dec 2024 23:07:32 GMT"} })
5969	2024-12-20T23:07:32.618Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5970	2024-12-20T23:07:32.618Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "24d40094-3050-4b17-b729-e9408aba718b", "content-length": "2", "date": "Fri, 20 Dec 2024 23:07:32 GMT"} })
5971	2024-12-20T23:07:32.619Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
5972	2024-12-20T23:07:32.619Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5973	2024-12-20T23:07:32.620Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5974	2024-12-20T23:07:32.621Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 237.300426ms iteration = 28 status = null
5975	2024-12-20T23:07:32.644Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5976	2024-12-20T23:07:32.644Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5977	2024-12-20T23:07:32.671Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5978	2024-12-20T23:07:32.672Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
5979	2024-12-20T23:07:32.672Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5980	2024-12-20T23:07:32.673Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5981	2024-12-20T23:07:32.676Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5982	2024-12-20T23:07:32.678Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
5983	2024-12-20T23:07:32.678Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5984	2024-12-20T23:07:32.678Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5985	2024-12-20T23:07:32.714Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5986	2024-12-20T23:07:32.714Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5987	2024-12-20T23:07:32.716Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 03dd12a0-dc2d-46e1-a7f4-d3397b9b7aa7, rack_id: b394d0c6-19a8-4136-9148-af6c47c1f8e0, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(560ca89c-7c0c-4921-8e5c-b57676f695b0) } rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
5988	2024-12-20T23:07:32.716Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5989	2024-12-20T23:07:32.716Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5990	2024-12-20T23:07:32.727Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
5991	2024-12-20T23:07:32.727Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "4ad19b4e-0cda-44a8-b351-f65718522017", "content-length": "134", "date": "Fri, 20 Dec 2024 23:07:32 GMT"} })
5992	2024-12-20T23:07:32.727Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
5993	2024-12-20T23:07:32.727Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 560ca89c-7c0c-4921-8e5c-b57676f695b0
5994	2024-12-20T23:07:32.727Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
5995	2024-12-20T23:07:32.727Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
5996	2024-12-20T23:07:32.727Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5997	2024-12-20T23:07:32.727Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5998	2024-12-20T23:07:32.727Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5999	2024-12-20T23:07:32.727Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
6000	2024-12-20T23:07:32.728Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
6001	2024-12-20T23:07:32.728Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
6002	2024-12-20T23:07:32.728Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
6003	2024-12-20T23:07:32.728Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6004	2024-12-20T23:07:32.728Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6005	2024-12-20T23:07:32.730Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
6006	2024-12-20T23:07:32.730Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "9701c831-77ff-4039-ab2f-7a9d0742478c", "content-length": "2", "date": "Fri, 20 Dec 2024 23:07:32 GMT"} })
6007	2024-12-20T23:07:32.730Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
6008	2024-12-20T23:07:32.730Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "05d3fc09-9d58-4a01-affc-efcad5f2f96c", "content-length": "52", "date": "Fri, 20 Dec 2024 23:07:32 GMT"} })
6009	2024-12-20T23:07:32.730Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
6010	2024-12-20T23:07:32.730Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
6011	2024-12-20T23:07:32.789Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
6012	2024-12-20T23:07:32.854Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "06510df4-6c66-4579-913f-c7cf40d8d66d", "date": "Fri, 20 Dec 2024 23:07:32 GMT"} })
6013	2024-12-20T23:07:32.854Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch = Switch0
6014	2024-12-20T23:07:32.854Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
6015	2024-12-20T23:07:32.854Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "b6900e15-4a37-4ae6-8c2b-1f8cce134e57", "content-length": "691", "date": "Fri, 20 Dec 2024 23:07:32 GMT"} })
6016	2024-12-20T23:07:32.854Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
6017	2024-12-20T23:07:32.854Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
6018	2024-12-20T23:07:32.854Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
6019	2024-12-20T23:07:32.854Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "f7f31516-5a32-4a18-a70f-767afa7386d6", "content-length": "682", "date": "Fri, 20 Dec 2024 23:07:32 GMT"} })
6020	2024-12-20T23:07:32.854Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6021	2024-12-20T23:07:32.855Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6022	2024-12-20T23:07:32.856Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 472.086072ms iteration = 28 status = {}
6023	2024-12-20T23:07:32.858Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6024	2024-12-20T23:07:32.858Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6025	2024-12-20T23:07:32.862Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6026	2024-12-20T23:07:32.862Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6027	2024-12-20T23:07:32.864Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6028	2024-12-20T23:07:32.864Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6029	2024-12-20T23:07:32.875Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 490.542889ms iteration = 28 status = {}
6030	2024-12-20T23:07:33.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 305 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 01747af0-8dc3-4524-8af2-c409c84842da response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
6031	2024-12-20T23:07:33.308Z	DEBG	nexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
6032	2024-12-20T23:07:33.308Z	DEBG	nexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
6033	2024-12-20T23:07:33.339Z	DEBG	nexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
6034	2024-12-20T23:07:33.339Z	INFO	nexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:63820
6035	2024-12-20T23:07:33.339Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:63820 req_id = 52a35ce9-3042-41b7-8374-85cc533126ea roles = RoleSet { roles: {} } uri = /metrics/producers
6036	2024-12-20T23:07:33.339Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:63820 req_id = 52a35ce9-3042-41b7-8374-85cc533126ea resource = Database result = Ok(()) uri = /metrics/producers
6037	2024-12-20T23:07:33.377Z	INFO	nexus: assigned collector to new producer collector_id = 103fc383-10f6-4327-a1b1-dabac4e1e365 file = nexus/src/app/oximeter.rs:76 producer_id = f3b60bd3-6e34-4140-a755-57f26dbc5b34
6038	2024-12-20T23:07:33.377Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 38277 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:63820 req_id = 52a35ce9-3042-41b7-8374-85cc533126ea response_code = 201 uri = /metrics/producers
6039	2024-12-20T23:07:33.377Z	DEBG	nexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "52a35ce9-3042-41b7-8374-85cc533126ea", "content-length": "41", "date": "Fri, 20 Dec 2024 23:07:33 GMT"} })
6040	2024-12-20T23:07:33.379Z	DEBG	nexus (producer-registration-task): registered with nexus successfully
6041	2024-12-20T23:07:33.379Z	DEBG	nexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
6042	2024-12-20T23:07:42.384Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 83 reason = Timeout
6043	2024-12-20T23:07:42.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 83 reason = Timeout
6044	2024-12-20T23:07:42.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6045	2024-12-20T23:07:42.384Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6046	2024-12-20T23:07:42.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6047	2024-12-20T23:07:42.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6048	2024-12-20T23:07:42.413Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6049	2024-12-20T23:07:42.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6050	2024-12-20T23:07:42.414Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6051	2024-12-20T23:07:42.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6052	2024-12-20T23:07:42.416Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6053	2024-12-20T23:07:42.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6054	2024-12-20T23:07:42.419Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6055	2024-12-20T23:07:42.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6056	2024-12-20T23:07:42.442Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 58.198398ms iteration = 83 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6057	2024-12-20T23:07:42.448Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6058	2024-12-20T23:07:42.450Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
6059	2024-12-20T23:07:42.450Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6060	2024-12-20T23:07:42.451Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6061	2024-12-20T23:07:42.463Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
6062	2024-12-20T23:07:42.463Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 78.477165ms iteration = 83 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
6063	2024-12-20T23:07:43.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 266 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 092e99ea-a185-47d0-bcd2-f1ba38ed9c64 response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
6064	2024-12-20T23:07:52.384Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 84 reason = Timeout
6065	2024-12-20T23:07:52.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 84 reason = Timeout
6066	2024-12-20T23:07:52.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6067	2024-12-20T23:07:52.384Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6068	2024-12-20T23:07:52.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6069	2024-12-20T23:07:52.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6070	2024-12-20T23:07:52.386Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6071	2024-12-20T23:07:52.387Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6072	2024-12-20T23:07:52.387Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6073	2024-12-20T23:07:52.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6074	2024-12-20T23:07:52.389Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6075	2024-12-20T23:07:52.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6076	2024-12-20T23:07:52.391Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6077	2024-12-20T23:07:52.391Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6078	2024-12-20T23:07:52.393Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6079	2024-12-20T23:07:52.395Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
6080	2024-12-20T23:07:52.395Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6081	2024-12-20T23:07:52.395Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6082	2024-12-20T23:07:52.407Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
6083	2024-12-20T23:07:52.407Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.841957ms iteration = 84 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
6084	2024-12-20T23:07:52.416Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.507315ms iteration = 84 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6085	2024-12-20T23:07:53.147Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 318 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = a095c7a5-a862-4863-bf5f-25824e57f0b0 response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
6086	2024-12-20T23:08:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 29 reason = Timeout
6087	2024-12-20T23:08:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
6088	2024-12-20T23:08:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 29 reason = Timeout
6089	2024-12-20T23:08:02.384Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
6090	2024-12-20T23:08:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 29 reason = Timeout
6091	2024-12-20T23:08:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6092	2024-12-20T23:08:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
6093	2024-12-20T23:08:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = external_endpoints iteration = 15 reason = Timeout
6094	2024-12-20T23:08:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 15 reason = Timeout
6095	2024-12-20T23:08:02.384Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
6096	2024-12-20T23:08:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6097	2024-12-20T23:08:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6098	2024-12-20T23:08:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 15 reason = Timeout
6099	2024-12-20T23:08:02.384Z	INFO	nexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
6100	2024-12-20T23:08:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
6101	2024-12-20T23:08:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 15 reason = Timeout
6102	2024-12-20T23:08:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 29 reason = Timeout
6103	2024-12-20T23:08:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 29 reason = Timeout
6104	2024-12-20T23:08:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
6105	2024-12-20T23:08:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 17 reason = Timeout
6106	2024-12-20T23:08:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 15 reason = Timeout
6107	2024-12-20T23:08:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
6108	2024-12-20T23:08:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 15 reason = Timeout
6109	2024-12-20T23:08:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
6110	2024-12-20T23:08:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 29 reason = Timeout
6111	2024-12-20T23:08:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 85 reason = Timeout
6112	2024-12-20T23:08:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6113	2024-12-20T23:08:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6114	2024-12-20T23:08:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 31 reason = Timeout
6115	2024-12-20T23:08:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 29 reason = Timeout
6116	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6117	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 29 reason = Timeout
6118	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
6119	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
6120	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 29 reason = Timeout
6121	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
6122	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 16 reason = Timeout
6123	2024-12-20T23:08:02.385Z	WARN	nexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7
6124	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 13.96\u{b5}s iteration = 16 status = {"enabled":false,"target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7"}
6125	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 85 reason = Timeout
6126	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 15 reason = Timeout
6127	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
6128	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 29 reason = Timeout
6129	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6130	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
6131	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 17 reason = Timeout
6132	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 29 reason = Timeout
6133	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6134	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 29 reason = Timeout
6135	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6136	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 29 reason = Timeout
6137	2024-12-20T23:08:02.385Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
6138	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6139	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 29 reason = Timeout
6140	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6141	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 15 reason = Timeout
6142	2024-12-20T23:08:02.385Z	INFO	nexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-20 22:58:02.385115276 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
6143	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
6144	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 15 reason = Timeout
6145	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
6146	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6147	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 15 reason = Timeout
6148	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
6149	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6150	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6151	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
6152	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
6153	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6154	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
6155	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
6156	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
6157	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6158	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6159	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
6160	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
6161	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6162	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6163	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
6164	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
6165	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6166	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6167	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6168	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6169	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
6170	2024-12-20T23:08:02.385Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("616ce87c-1f39-49d9-ba35-ee63b2650ecd.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b4615881-2f68-4c82-be17-071bf3ae3641.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c9217e92-0af3-4930-acb2-aa82ded5b782.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("c9217e92-0af3-4930-acb2-aa82ded5b782.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 1497, tv_nsec: 775897789 } })
6171	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6172	2024-12-20T23:08:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6173	2024-12-20T23:08:02.385Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1e296b8d-6800-4056-8034-a1ec2d005fff.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("ace0592c-7755-46f8-b47a-c9b6444fe64c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("ace0592c-7755-46f8-b47a-c9b6444fe64c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1497, tv_nsec: 775997542 } })
6174	2024-12-20T23:08:02.386Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
6175	2024-12-20T23:08:02.386Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 901.934\u{b5}s iteration = 15 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
6176	2024-12-20T23:08:02.386Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
6177	2024-12-20T23:08:02.386Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.444819ms iteration = 15 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
6178	2024-12-20T23:08:02.387Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6179	2024-12-20T23:08:02.387Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6180	2024-12-20T23:08:02.388Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.323751ms iteration = 29 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
6181	2024-12-20T23:08:02.388Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1497, tv_nsec: 778684056 } })
6182	2024-12-20T23:08:02.388Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
6183	2024-12-20T23:08:02.390Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6184	2024-12-20T23:08:02.391Z	DEBG	nexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 6.836567ms iteration = 15 status = {"errors":[],"found_port_ok":[]}
6185	2024-12-20T23:08:02.392Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
6186	2024-12-20T23:08:02.392Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6187	2024-12-20T23:08:02.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6188	2024-12-20T23:08:02.394Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6189	2024-12-20T23:08:02.398Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
6190	2024-12-20T23:08:02.398Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
6191	2024-12-20T23:08:02.398Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
6192	2024-12-20T23:08:02.404Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6193	2024-12-20T23:08:02.404Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6194	2024-12-20T23:08:02.405Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1497, tv_nsec: 795841577 } })
6195	2024-12-20T23:08:02.405Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
6196	2024-12-20T23:08:02.406Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6197	2024-12-20T23:08:02.406Z	INFO	nexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-20 22:58:02.385115276 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
6198	2024-12-20T23:08:02.406Z	DEBG	nexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 21.760077ms iteration = 15 status = {"expiration":"2024-12-20T22:58:02.385115276Z","pruned":[]}
6199	2024-12-20T23:08:02.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
6200	2024-12-20T23:08:02.411Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6201	2024-12-20T23:08:02.411Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6202	2024-12-20T23:08:02.413Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6203	2024-12-20T23:08:02.413Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6204	2024-12-20T23:08:02.418Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
6205	2024-12-20T23:08:02.418Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 33.83606ms iteration = 29 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
6206	2024-12-20T23:08:02.420Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6207	2024-12-20T23:08:02.421Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6208	2024-12-20T23:08:02.429Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6209	2024-12-20T23:08:02.431Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6210	2024-12-20T23:08:02.431Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
6211	2024-12-20T23:08:02.431Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6212	2024-12-20T23:08:02.432Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
6213	2024-12-20T23:08:02.432Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
6214	2024-12-20T23:08:02.432Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6215	2024-12-20T23:08:02.432Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
6216	2024-12-20T23:08:02.432Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
6217	2024-12-20T23:08:02.432Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
6218	2024-12-20T23:08:02.435Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6219	2024-12-20T23:08:02.435Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6220	2024-12-20T23:08:02.435Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6221	2024-12-20T23:08:02.439Z	DEBG	nexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
6222	2024-12-20T23:08:02.439Z	INFO	nexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
6223	2024-12-20T23:08:02.439Z	DEBG	nexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 51.756571ms iteration = 15 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
6224	2024-12-20T23:08:02.439Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
6225	2024-12-20T23:08:02.439Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6226	2024-12-20T23:08:02.439Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6227	2024-12-20T23:08:02.445Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6228	2024-12-20T23:08:02.445Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6229	2024-12-20T23:08:02.445Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6230	2024-12-20T23:08:02.445Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
6231	2024-12-20T23:08:02.445Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6232	2024-12-20T23:08:02.445Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6233	2024-12-20T23:08:02.445Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6234	2024-12-20T23:08:02.445Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
6235	2024-12-20T23:08:02.445Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6236	2024-12-20T23:08:02.445Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 60.711707ms iteration = 29 status = {"errors":[],"finish_invoked_ok":[]}
6237	2024-12-20T23:08:02.445Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
6238	2024-12-20T23:08:02.448Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
6239	2024-12-20T23:08:02.448Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6240	2024-12-20T23:08:02.448Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6241	2024-12-20T23:08:02.448Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
6242	2024-12-20T23:08:02.448Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6243	2024-12-20T23:08:02.448Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6244	2024-12-20T23:08:02.451Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 66.879556ms iteration = 29 status = {"errors":[],"garbage_collect_requested":[]}
6245	2024-12-20T23:08:02.453Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6246	2024-12-20T23:08:02.453Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6247	2024-12-20T23:08:02.453Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6248	2024-12-20T23:08:02.454Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1497, tv_nsec: 844396690 } })
6249	2024-12-20T23:08:02.454Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
6250	2024-12-20T23:08:02.455Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
6251	2024-12-20T23:08:02.455Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6252	2024-12-20T23:08:02.455Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6253	2024-12-20T23:08:02.455Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6254	2024-12-20T23:08:02.458Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
6255	2024-12-20T23:08:02.459Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6256	2024-12-20T23:08:02.461Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6257	2024-12-20T23:08:02.461Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
6258	2024-12-20T23:08:02.465Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6259	2024-12-20T23:08:02.465Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6260	2024-12-20T23:08:02.466Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6261	2024-12-20T23:08:02.467Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6262	2024-12-20T23:08:02.467Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6263	2024-12-20T23:08:02.467Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6264	2024-12-20T23:08:02.467Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6265	2024-12-20T23:08:02.469Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6266	2024-12-20T23:08:02.470Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6267	2024-12-20T23:08:02.471Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6268	2024-12-20T23:08:02.473Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
6269	2024-12-20T23:08:02.473Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6270	2024-12-20T23:08:02.473Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6271	2024-12-20T23:08:02.475Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6272	2024-12-20T23:08:02.476Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
6273	2024-12-20T23:08:02.477Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
6274	2024-12-20T23:08:02.478Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 93.627511ms iteration = 29 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
6275	2024-12-20T23:08:02.478Z	DEBG	nexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
6276	2024-12-20T23:08:02.478Z	INFO	nexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
6277	2024-12-20T23:08:02.478Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 287d6774-d3e6-4b73-8c77-c90534d4a54d, lookup_type: ById(287d6774-d3e6-4b73-8c77-c90534d4a54d) } result = Ok(())
6278	2024-12-20T23:08:02.478Z	DEBG	nexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 94.081514ms iteration = 15 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
6279	2024-12-20T23:08:02.478Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6280	2024-12-20T23:08:02.479Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6281	2024-12-20T23:08:02.481Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 96.261034ms iteration = 29 status = 0
6282	2024-12-20T23:08:02.484Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 99.764229ms iteration = 85 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6283	2024-12-20T23:08:02.485Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6284	2024-12-20T23:08:02.486Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6285	2024-12-20T23:08:02.486Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
6286	2024-12-20T23:08:02.486Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 101.385684ms iteration = 85 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
6287	2024-12-20T23:08:02.495Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
6288	2024-12-20T23:08:02.495Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
6289	2024-12-20T23:08:02.495Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
6290	2024-12-20T23:08:02.495Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6291	2024-12-20T23:08:02.495Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
6292	2024-12-20T23:08:02.495Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
6293	2024-12-20T23:08:02.495Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
6294	2024-12-20T23:08:02.495Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6295	2024-12-20T23:08:02.495Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6296	2024-12-20T23:08:02.495Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6297	2024-12-20T23:08:02.497Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6298	2024-12-20T23:08:02.497Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
6299	2024-12-20T23:08:02.497Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6300	2024-12-20T23:08:02.497Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6301	2024-12-20T23:08:02.497Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6302	2024-12-20T23:08:02.497Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6303	2024-12-20T23:08:02.498Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 113.474306ms iteration = 29 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
6304	2024-12-20T23:08:02.498Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6305	2024-12-20T23:08:02.500Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6306	2024-12-20T23:08:02.500Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6307	2024-12-20T23:08:02.500Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6308	2024-12-20T23:08:02.500Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
6309	2024-12-20T23:08:02.500Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 116.048747ms iteration = 29 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
6310	2024-12-20T23:08:02.501Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
6311	2024-12-20T23:08:02.501Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6312	2024-12-20T23:08:02.501Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6313	2024-12-20T23:08:02.501Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
6314	2024-12-20T23:08:02.501Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6315	2024-12-20T23:08:02.502Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6316	2024-12-20T23:08:02.503Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
6317	2024-12-20T23:08:02.503Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6318	2024-12-20T23:08:02.503Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6319	2024-12-20T23:08:02.528Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 143.14541ms iteration = 29 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
6320	2024-12-20T23:08:02.534Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
6321	2024-12-20T23:08:02.535Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
6322	2024-12-20T23:08:02.536Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
6323	2024-12-20T23:08:02.536Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 5fa20c62-75c0-4531-be07-51b87d303d18 dns_zone_name = oxide.test found_so_far = 0 version = 2
6324	2024-12-20T23:08:02.536Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6325	2024-12-20T23:08:02.536Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6326	2024-12-20T23:08:02.538Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6327	2024-12-20T23:08:02.539Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6328	2024-12-20T23:08:02.540Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 155.473519ms iteration = 31 status = {"physical_disks_added":0}
6329	2024-12-20T23:08:02.541Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
6330	2024-12-20T23:08:02.541Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6331	2024-12-20T23:08:02.541Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6332	2024-12-20T23:08:02.541Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
6333	2024-12-20T23:08:02.541Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6334	2024-12-20T23:08:02.541Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
6335	2024-12-20T23:08:02.541Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 1e86bf23-6ffd-4a12-93c7-0061d7d17590 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
6336	2024-12-20T23:08:02.541Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6337	2024-12-20T23:08:02.541Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6338	2024-12-20T23:08:02.542Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6339	2024-12-20T23:08:02.543Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 5fa20c62-75c0-4531-be07-51b87d303d18 dns_zone_name = oxide.test found_so_far = 1 version = 2
6340	2024-12-20T23:08:02.543Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
6341	2024-12-20T23:08:02.543Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-20 22:54:39.899407 UTC dns_group = external generation = 2
6342	2024-12-20T23:08:02.543Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 158.142723ms iteration = 15 status = {"generation":2}
6343	2024-12-20T23:08:02.544Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6344	2024-12-20T23:08:02.546Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
6345	2024-12-20T23:08:02.546Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6346	2024-12-20T23:08:02.547Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6347	2024-12-20T23:08:02.549Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 1e86bf23-6ffd-4a12-93c7-0061d7d17590 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
6348	2024-12-20T23:08:02.549Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
6349	2024-12-20T23:08:02.549Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-20 22:54:38.956071 UTC dns_group = internal generation = 1
6350	2024-12-20T23:08:02.549Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 164.59676ms iteration = 15 status = {"generation":1}
6351	2024-12-20T23:08:02.551Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6352	2024-12-20T23:08:02.552Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6353	2024-12-20T23:08:02.557Z	WARN	nexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
6354	2024-12-20T23:08:02.557Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 172.191318ms iteration = 15 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
6355	2024-12-20T23:08:02.578Z	DEBG	nexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-20T22:53:58.073373Z, time_modified: 2024-12-20T22:53:58.073373Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, db_silo: Silo { identity: SiloIdentity { id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-20T22:54:39.847015Z, time_modified: 2024-12-20T22:54:39.847015Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, db_silo: Silo { identity: SiloIdentity { id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-20T22:54:39.847015Z, time_modified: 2024-12-20T22:54:39.847015Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156" }] }) }
6356	2024-12-20T23:08:02.578Z	DEBG	nexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 194.073938ms iteration = 15 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"ee579470-b16b-4af1-ad3a-b3d29f9a2fb2","tls_certs":["ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156"]}},"default_endpoint":{"silo_id":"ee579470-b16b-4af1-ad3a-b3d29f9a2fb2","tls_certs":["ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
6357	2024-12-20T23:08:02.630Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 zone_address = fd00:1122:3344:101::2
6358	2024-12-20T23:08:02.630Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6359	2024-12-20T23:08:02.630Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d3cdbc84-3f32-461d-8011-6c0602b0c923", "content-length": "26", "date": "Fri, 20 Dec 2024 23:08:02 GMT"} })
6360	2024-12-20T23:08:02.630Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6361	2024-12-20T23:08:02.632Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
6362	2024-12-20T23:08:02.636Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
6363	2024-12-20T23:08:02.636Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6364	2024-12-20T23:08:02.636Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0d80404-ce52-40a4-a917-ab7165b53592", "content-length": "26", "date": "Fri, 20 Dec 2024 23:08:02 GMT"} })
6365	2024-12-20T23:08:02.636Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6366	2024-12-20T23:08:02.639Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
6367	2024-12-20T23:08:02.646Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6368	2024-12-20T23:08:02.646Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6369	2024-12-20T23:08:02.672Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
6370	2024-12-20T23:08:02.672Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6371	2024-12-20T23:08:02.672Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "990f1d73-9832-4682-82d8-6a3eb4c00ff1", "content-length": "26", "date": "Fri, 20 Dec 2024 23:08:02 GMT"} })
6372	2024-12-20T23:08:02.672Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6373	2024-12-20T23:08:02.675Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
6374	2024-12-20T23:08:02.677Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6375	2024-12-20T23:08:02.679Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
6376	2024-12-20T23:08:02.679Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6377	2024-12-20T23:08:02.679Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6378	2024-12-20T23:08:02.712Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6379	2024-12-20T23:08:02.714Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
6380	2024-12-20T23:08:02.714Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6381	2024-12-20T23:08:02.714Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6382	2024-12-20T23:08:02.720Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
6383	2024-12-20T23:08:02.805Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
6384	2024-12-20T23:08:02.805Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "e9155ff4-2bd3-43c7-92f8-09f799d36ddd", "date": "Fri, 20 Dec 2024 23:08:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
6385	2024-12-20T23:08:02.805Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
6386	2024-12-20T23:08:02.808Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "a25b0751-7730-406f-a1fc-b77ba696d787", "date": "Fri, 20 Dec 2024 23:08:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
6387	2024-12-20T23:08:02.808Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
6388	2024-12-20T23:08:02.809Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
6389	2024-12-20T23:08:02.809Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 424.870976ms iteration = 17 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
6390	2024-12-20T23:08:02.816Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
6391	2024-12-20T23:08:02.816Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
6392	2024-12-20T23:08:02.817Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2947d8d-98eb-423b-a874-3a22adc89601", "content-length": "372", "date": "Fri, 20 Dec 2024 23:08:02 GMT"} })
6393	2024-12-20T23:08:02.817Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6394	2024-12-20T23:08:02.817Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6395	2024-12-20T23:08:02.841Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
6396	2024-12-20T23:08:02.841Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
6397	2024-12-20T23:08:02.841Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
6398	2024-12-20T23:08:02.842Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c2e85ec-582c-40e5-935a-52418dd7c74f", "content-length": "2", "date": "Fri, 20 Dec 2024 23:08:02 GMT"} })
6399	2024-12-20T23:08:02.842Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "6ff7ce21-7a11-4764-810c-0dfc0c2edb12", "date": "Fri, 20 Dec 2024 23:08:02 GMT"} })
6400	2024-12-20T23:08:02.842Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6401	2024-12-20T23:08:02.842Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6402	2024-12-20T23:08:02.843Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
6403	2024-12-20T23:08:02.843Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6404	2024-12-20T23:08:02.844Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6405	2024-12-20T23:08:02.845Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6406	2024-12-20T23:08:02.845Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 460.680389ms iteration = 29 status = null
6407	2024-12-20T23:08:02.848Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6408	2024-12-20T23:08:02.848Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6409	2024-12-20T23:08:02.852Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
6410	2024-12-20T23:08:02.852Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
6411	2024-12-20T23:08:02.853Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "8d1a16b3-eb9d-49b4-bd5a-eaab908e4f44", "date": "Fri, 20 Dec 2024 23:08:02 GMT"} })
6412	2024-12-20T23:08:02.855Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 470.428917ms iteration = 29 status = {}
6413	2024-12-20T23:08:02.864Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 479.628809ms iteration = 29 status = {}
6414	2024-12-20T23:08:02.865Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
6415	2024-12-20T23:08:02.865Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "394a17c8-2188-4467-abcc-baa2766e36f7", "date": "Fri, 20 Dec 2024 23:08:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
6416	2024-12-20T23:08:02.865Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
6417	2024-12-20T23:08:02.867Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "5cb3e11a-dcc6-4753-a47f-33a445da5152", "date": "Fri, 20 Dec 2024 23:08:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
6418	2024-12-20T23:08:02.867Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
6419	2024-12-20T23:08:02.870Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "199cfc06-7de1-49a5-8d8a-72b55fc2a596", "date": "Fri, 20 Dec 2024 23:08:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
6420	2024-12-20T23:08:02.870Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
6421	2024-12-20T23:08:02.872Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
6422	2024-12-20T23:08:02.872Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 487.456425ms iteration = 17 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
6423	2024-12-20T23:08:02.958Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
6424	2024-12-20T23:08:02.958Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "1f9acafb-5ee5-4a97-b984-7f08483c3078", "content-length": "691", "date": "Fri, 20 Dec 2024 23:08:02 GMT"} })
6425	2024-12-20T23:08:02.958Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
6426	2024-12-20T23:08:02.958Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
6427	2024-12-20T23:08:02.958Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
6428	2024-12-20T23:08:02.959Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 574.509805ms iteration = 29 status = {}
6429	2024-12-20T23:08:02.968Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6430	2024-12-20T23:08:02.969Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6431	2024-12-20T23:08:02.971Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 03dd12a0-dc2d-46e1-a7f4-d3397b9b7aa7, rack_id: b394d0c6-19a8-4136-9148-af6c47c1f8e0, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(560ca89c-7c0c-4921-8e5c-b57676f695b0) } rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
6432	2024-12-20T23:08:02.971Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6433	2024-12-20T23:08:02.971Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6434	2024-12-20T23:08:02.980Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
6435	2024-12-20T23:08:02.980Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "b8c7fe5f-7fea-45f5-8994-cd842a20238f", "content-length": "134", "date": "Fri, 20 Dec 2024 23:08:02 GMT"} })
6436	2024-12-20T23:08:02.980Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
6437	2024-12-20T23:08:02.980Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 560ca89c-7c0c-4921-8e5c-b57676f695b0
6438	2024-12-20T23:08:02.980Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
6439	2024-12-20T23:08:02.980Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
6440	2024-12-20T23:08:02.980Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
6441	2024-12-20T23:08:02.980Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
6442	2024-12-20T23:08:02.980Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
6443	2024-12-20T23:08:02.980Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
6444	2024-12-20T23:08:02.981Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
6445	2024-12-20T23:08:02.981Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
6446	2024-12-20T23:08:02.981Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
6447	2024-12-20T23:08:02.981Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6448	2024-12-20T23:08:02.981Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6449	2024-12-20T23:08:02.984Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
6450	2024-12-20T23:08:02.985Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "11eda893-6abd-4a1f-87fb-cfe88e1ffb1e", "content-length": "2", "date": "Fri, 20 Dec 2024 23:08:02 GMT"} })
6451	2024-12-20T23:08:02.985Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
6452	2024-12-20T23:08:02.985Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "8c50f94b-af60-48bb-8e9e-f8a15f21ec60", "content-length": "52", "date": "Fri, 20 Dec 2024 23:08:02 GMT"} })
6453	2024-12-20T23:08:02.985Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
6454	2024-12-20T23:08:02.985Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
6455	2024-12-20T23:08:03.091Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "4a98c06a-267b-499c-9695-b64f97e16737", "date": "Fri, 20 Dec 2024 23:08:02 GMT"} })
6456	2024-12-20T23:08:03.091Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch = Switch0
6457	2024-12-20T23:08:03.091Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
6458	2024-12-20T23:08:03.091Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "b887f39e-e95a-4964-ba65-46539d2ca889", "content-length": "682", "date": "Fri, 20 Dec 2024 23:08:03 GMT"} })
6459	2024-12-20T23:08:03.091Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6460	2024-12-20T23:08:03.092Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6461	2024-12-20T23:08:03.095Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6462	2024-12-20T23:08:03.096Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6463	2024-12-20T23:08:03.099Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6464	2024-12-20T23:08:03.100Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6465	2024-12-20T23:08:03.102Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6466	2024-12-20T23:08:03.102Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6467	2024-12-20T23:08:03.109Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 724.293758ms iteration = 29 status = {}
6468	2024-12-20T23:08:03.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 299 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 8ff493a4-1aec-45e9-ae63-3bd6632c8719 response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
6469	2024-12-20T23:08:12.384Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 86 reason = Timeout
6470	2024-12-20T23:08:12.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 86 reason = Timeout
6471	2024-12-20T23:08:12.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6472	2024-12-20T23:08:12.384Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6473	2024-12-20T23:08:12.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6474	2024-12-20T23:08:12.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6475	2024-12-20T23:08:12.414Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6476	2024-12-20T23:08:12.414Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6477	2024-12-20T23:08:12.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6478	2024-12-20T23:08:12.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6479	2024-12-20T23:08:12.416Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6480	2024-12-20T23:08:12.416Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.107352ms iteration = 86 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6481	2024-12-20T23:08:12.416Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6482	2024-12-20T23:08:12.419Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6483	2024-12-20T23:08:12.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6484	2024-12-20T23:08:12.420Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6485	2024-12-20T23:08:12.423Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
6486	2024-12-20T23:08:12.423Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6487	2024-12-20T23:08:12.423Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6488	2024-12-20T23:08:12.434Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
6489	2024-12-20T23:08:12.434Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 50.24079ms iteration = 86 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
6490	2024-12-20T23:08:13.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 308 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 705cf38f-9bfb-49b4-a3a3-88d03846078c response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
6491	2024-12-20T23:08:22.384Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 87 reason = Timeout
6492	2024-12-20T23:08:22.384Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6493	2024-12-20T23:08:22.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 87 reason = Timeout
6494	2024-12-20T23:08:22.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6495	2024-12-20T23:08:22.384Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6496	2024-12-20T23:08:22.384Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6497	2024-12-20T23:08:22.386Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6498	2024-12-20T23:08:22.386Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6499	2024-12-20T23:08:22.387Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6500	2024-12-20T23:08:22.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6501	2024-12-20T23:08:22.389Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6502	2024-12-20T23:08:22.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6503	2024-12-20T23:08:22.391Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6504	2024-12-20T23:08:22.391Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6505	2024-12-20T23:08:22.393Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6506	2024-12-20T23:08:22.395Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
6507	2024-12-20T23:08:22.395Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6508	2024-12-20T23:08:22.395Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6509	2024-12-20T23:08:22.406Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
6510	2024-12-20T23:08:22.406Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.327704ms iteration = 87 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
6511	2024-12-20T23:08:22.421Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 37.439048ms iteration = 87 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6512	2024-12-20T23:08:23.147Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 315 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = d46fdb42-8190-4aaf-bb5c-2da5a250323c response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
6513	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 30 reason = Timeout
6514	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
6515	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 30 reason = Timeout
6516	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6517	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 30 reason = Timeout
6518	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 30 reason = Timeout
6519	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
6520	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 30 reason = Timeout
6521	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6522	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6523	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 30 reason = Timeout
6524	2024-12-20T23:08:32.385Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
6525	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 88 reason = Timeout
6526	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6527	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 32 reason = Timeout
6528	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6529	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6530	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 88 reason = Timeout
6531	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6532	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 30 reason = Timeout
6533	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
6534	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 30 reason = Timeout
6535	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6536	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 30 reason = Timeout
6537	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
6538	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 30 reason = Timeout
6539	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
6540	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 30 reason = Timeout
6541	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
6542	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 30 reason = Timeout
6543	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
6544	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6545	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 30 reason = Timeout
6546	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 30 reason = Timeout
6547	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
6548	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
6549	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6550	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6551	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6552	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
6553	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6554	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6555	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
6556	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6557	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6558	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6559	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
6560	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
6561	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6562	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
6563	2024-12-20T23:08:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6564	2024-12-20T23:08:32.388Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6565	2024-12-20T23:08:32.388Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1527, tv_nsec: 778569763 } })
6566	2024-12-20T23:08:32.388Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6567	2024-12-20T23:08:32.388Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
6568	2024-12-20T23:08:32.389Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.983739ms iteration = 30 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
6569	2024-12-20T23:08:32.389Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6570	2024-12-20T23:08:32.391Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
6571	2024-12-20T23:08:32.391Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6572	2024-12-20T23:08:32.391Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6573	2024-12-20T23:08:32.391Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
6574	2024-12-20T23:08:32.391Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6575	2024-12-20T23:08:32.391Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6576	2024-12-20T23:08:32.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6577	2024-12-20T23:08:32.392Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6578	2024-12-20T23:08:32.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6579	2024-12-20T23:08:32.394Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6580	2024-12-20T23:08:32.394Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6581	2024-12-20T23:08:32.395Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 10.109448ms iteration = 30 status = {"errors":[],"finish_invoked_ok":[]}
6582	2024-12-20T23:08:32.395Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6583	2024-12-20T23:08:32.396Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
6584	2024-12-20T23:08:32.396Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6585	2024-12-20T23:08:32.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6586	2024-12-20T23:08:32.397Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
6587	2024-12-20T23:08:32.397Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6588	2024-12-20T23:08:32.397Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6589	2024-12-20T23:08:32.397Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
6590	2024-12-20T23:08:32.398Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
6591	2024-12-20T23:08:32.398Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
6592	2024-12-20T23:08:32.399Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1527, tv_nsec: 789222916 } })
6593	2024-12-20T23:08:32.399Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
6594	2024-12-20T23:08:32.400Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6595	2024-12-20T23:08:32.400Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6596	2024-12-20T23:08:32.400Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6597	2024-12-20T23:08:32.400Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1527, tv_nsec: 791078446 } })
6598	2024-12-20T23:08:32.401Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6599	2024-12-20T23:08:32.401Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
6600	2024-12-20T23:08:32.402Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6601	2024-12-20T23:08:32.402Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6602	2024-12-20T23:08:32.402Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6603	2024-12-20T23:08:32.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
6604	2024-12-20T23:08:32.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6605	2024-12-20T23:08:32.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6606	2024-12-20T23:08:32.423Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6607	2024-12-20T23:08:32.423Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6608	2024-12-20T23:08:32.424Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
6609	2024-12-20T23:08:32.424Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 39.69282ms iteration = 88 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
6610	2024-12-20T23:08:32.425Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6611	2024-12-20T23:08:32.425Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6612	2024-12-20T23:08:32.426Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 41.110419ms iteration = 30 status = {"errors":[],"garbage_collect_requested":[]}
6613	2024-12-20T23:08:32.426Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
6614	2024-12-20T23:08:32.427Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6615	2024-12-20T23:08:32.427Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
6616	2024-12-20T23:08:32.428Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6617	2024-12-20T23:08:32.428Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6618	2024-12-20T23:08:32.428Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 43.557816ms iteration = 30 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
6619	2024-12-20T23:08:32.429Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 287d6774-d3e6-4b73-8c77-c90534d4a54d, lookup_type: ById(287d6774-d3e6-4b73-8c77-c90534d4a54d) } result = Ok(())
6620	2024-12-20T23:08:32.429Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6621	2024-12-20T23:08:32.429Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6622	2024-12-20T23:08:32.429Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6623	2024-12-20T23:08:32.430Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6624	2024-12-20T23:08:32.431Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 46.402145ms iteration = 30 status = 0
6625	2024-12-20T23:08:32.434Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6626	2024-12-20T23:08:32.434Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6627	2024-12-20T23:08:32.435Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 50.788614ms iteration = 88 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6628	2024-12-20T23:08:32.437Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6629	2024-12-20T23:08:32.437Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
6630	2024-12-20T23:08:32.437Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6631	2024-12-20T23:08:32.438Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6632	2024-12-20T23:08:32.440Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6633	2024-12-20T23:08:32.441Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
6634	2024-12-20T23:08:32.441Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
6635	2024-12-20T23:08:32.442Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
6636	2024-12-20T23:08:32.442Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6637	2024-12-20T23:08:32.442Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6638	2024-12-20T23:08:32.446Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
6639	2024-12-20T23:08:32.446Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 61.078377ms iteration = 30 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
6640	2024-12-20T23:08:32.446Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6641	2024-12-20T23:08:32.449Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
6642	2024-12-20T23:08:32.449Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6643	2024-12-20T23:08:32.449Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6644	2024-12-20T23:08:32.462Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
6645	2024-12-20T23:08:32.462Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6646	2024-12-20T23:08:32.462Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f67edb6-56b3-44f7-92df-2448c03f5207", "content-length": "26", "date": "Fri, 20 Dec 2024 23:08:32 GMT"} })
6647	2024-12-20T23:08:32.462Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6648	2024-12-20T23:08:32.464Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
6649	2024-12-20T23:08:32.465Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
6650	2024-12-20T23:08:32.465Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6651	2024-12-20T23:08:32.466Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "264a642e-af6d-4a4e-b263-4ae5b302da8e", "content-length": "26", "date": "Fri, 20 Dec 2024 23:08:32 GMT"} })
6652	2024-12-20T23:08:32.466Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6653	2024-12-20T23:08:32.467Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
6654	2024-12-20T23:08:32.467Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6655	2024-12-20T23:08:32.467Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
6656	2024-12-20T23:08:32.468Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6657	2024-12-20T23:08:32.468Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6658	2024-12-20T23:08:32.473Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 87.946375ms iteration = 30 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
6659	2024-12-20T23:08:32.475Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 zone_address = fd00:1122:3344:101::2
6660	2024-12-20T23:08:32.475Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6661	2024-12-20T23:08:32.475Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 90.7085ms iteration = 30 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
6662	2024-12-20T23:08:32.475Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "16a1f3aa-752c-4738-8151-19a646b6ebd5", "content-length": "26", "date": "Fri, 20 Dec 2024 23:08:32 GMT"} })
6663	2024-12-20T23:08:32.476Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6664	2024-12-20T23:08:32.477Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
6665	2024-12-20T23:08:32.484Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 98.905775ms iteration = 32 status = {"physical_disks_added":0}
6666	2024-12-20T23:08:32.497Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6667	2024-12-20T23:08:32.497Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
6668	2024-12-20T23:08:32.497Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 112.292763ms iteration = 30 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
6669	2024-12-20T23:08:32.500Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6670	2024-12-20T23:08:32.500Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6671	2024-12-20T23:08:32.543Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 158.566064ms iteration = 30 status = {}
6672	2024-12-20T23:08:32.550Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
6673	2024-12-20T23:08:32.550Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e3dd14c-ad77-4959-a9a8-29e1bf413775", "content-length": "2", "date": "Fri, 20 Dec 2024 23:08:32 GMT"} })
6674	2024-12-20T23:08:32.551Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
6675	2024-12-20T23:08:32.551Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6676	2024-12-20T23:08:32.553Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 168.514507ms iteration = 30 status = null
6677	2024-12-20T23:08:32.562Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6678	2024-12-20T23:08:32.562Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6679	2024-12-20T23:08:32.572Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
6680	2024-12-20T23:08:32.572Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
6681	2024-12-20T23:08:32.573Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "eda82669-2971-4de5-84b4-de2eb9ea9842", "content-length": "372", "date": "Fri, 20 Dec 2024 23:08:32 GMT"} })
6682	2024-12-20T23:08:32.573Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6683	2024-12-20T23:08:32.573Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6684	2024-12-20T23:08:32.573Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6685	2024-12-20T23:08:32.578Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
6686	2024-12-20T23:08:32.578Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6687	2024-12-20T23:08:32.578Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6688	2024-12-20T23:08:32.580Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6689	2024-12-20T23:08:32.581Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
6690	2024-12-20T23:08:32.581Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6691	2024-12-20T23:08:32.581Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6692	2024-12-20T23:08:32.591Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
6693	2024-12-20T23:08:32.591Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
6694	2024-12-20T23:08:32.592Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "8c8b74e5-3445-430f-a5c7-66df42523052", "date": "Fri, 20 Dec 2024 23:08:32 GMT"} })
6695	2024-12-20T23:08:32.592Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6696	2024-12-20T23:08:32.592Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6697	2024-12-20T23:08:32.594Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6698	2024-12-20T23:08:32.594Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6699	2024-12-20T23:08:32.597Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6700	2024-12-20T23:08:32.597Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6701	2024-12-20T23:08:32.599Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
6702	2024-12-20T23:08:32.599Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
6703	2024-12-20T23:08:32.599Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "c264304c-4535-4fa7-8761-504d83ad1217", "date": "Fri, 20 Dec 2024 23:08:32 GMT"} })
6704	2024-12-20T23:08:32.601Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 216.381651ms iteration = 30 status = {}
6705	2024-12-20T23:08:32.623Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
6706	2024-12-20T23:08:32.623Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "ed839dca-fcef-441e-ab90-343455b27159", "content-length": "691", "date": "Fri, 20 Dec 2024 23:08:32 GMT"} })
6707	2024-12-20T23:08:32.624Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
6708	2024-12-20T23:08:32.624Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
6709	2024-12-20T23:08:32.624Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
6710	2024-12-20T23:08:32.625Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 240.177505ms iteration = 30 status = {}
6711	2024-12-20T23:08:32.630Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6712	2024-12-20T23:08:32.631Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6713	2024-12-20T23:08:32.632Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 03dd12a0-dc2d-46e1-a7f4-d3397b9b7aa7, rack_id: b394d0c6-19a8-4136-9148-af6c47c1f8e0, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(560ca89c-7c0c-4921-8e5c-b57676f695b0) } rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
6714	2024-12-20T23:08:32.633Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6715	2024-12-20T23:08:32.633Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6716	2024-12-20T23:08:32.641Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
6717	2024-12-20T23:08:32.642Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "e4cbb642-fd67-432c-a882-32a122b70e7b", "content-length": "134", "date": "Fri, 20 Dec 2024 23:08:32 GMT"} })
6718	2024-12-20T23:08:32.642Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
6719	2024-12-20T23:08:32.642Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 560ca89c-7c0c-4921-8e5c-b57676f695b0
6720	2024-12-20T23:08:32.642Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
6721	2024-12-20T23:08:32.642Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
6722	2024-12-20T23:08:32.642Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
6723	2024-12-20T23:08:32.642Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
6724	2024-12-20T23:08:32.642Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
6725	2024-12-20T23:08:32.642Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
6726	2024-12-20T23:08:32.642Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
6727	2024-12-20T23:08:32.642Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
6728	2024-12-20T23:08:32.642Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
6729	2024-12-20T23:08:32.642Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6730	2024-12-20T23:08:32.642Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6731	2024-12-20T23:08:32.645Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
6732	2024-12-20T23:08:32.645Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "e11c30d5-44b7-4ae9-810a-d2b4933381c8", "content-length": "2", "date": "Fri, 20 Dec 2024 23:08:32 GMT"} })
6733	2024-12-20T23:08:32.645Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
6734	2024-12-20T23:08:32.645Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "4b89ed80-4577-4026-90b4-d3d9ff31a2be", "content-length": "52", "date": "Fri, 20 Dec 2024 23:08:32 GMT"} })
6735	2024-12-20T23:08:32.645Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
6736	2024-12-20T23:08:32.645Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
6737	2024-12-20T23:08:32.759Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "d0a0177a-f65e-424a-bcfe-dc6fa7a9af88", "date": "Fri, 20 Dec 2024 23:08:32 GMT"} })
6738	2024-12-20T23:08:32.759Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch = Switch0
6739	2024-12-20T23:08:32.759Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
6740	2024-12-20T23:08:32.762Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "c47f3704-6f4d-4289-9973-efce2632a755", "content-length": "682", "date": "Fri, 20 Dec 2024 23:08:32 GMT"} })
6741	2024-12-20T23:08:32.762Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6742	2024-12-20T23:08:32.763Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6743	2024-12-20T23:08:32.766Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6744	2024-12-20T23:08:32.766Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6745	2024-12-20T23:08:32.771Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6746	2024-12-20T23:08:32.771Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6747	2024-12-20T23:08:32.773Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6748	2024-12-20T23:08:32.773Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6749	2024-12-20T23:08:32.780Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 395.910821ms iteration = 30 status = {}
6750	2024-12-20T23:08:33.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 292 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 5e51c5fe-72ae-440e-9f3c-7bcad039a14d response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
6751	2024-12-20T23:08:42.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 89 reason = Timeout
6752	2024-12-20T23:08:42.384Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 89 reason = Timeout
6753	2024-12-20T23:08:42.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6754	2024-12-20T23:08:42.384Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6755	2024-12-20T23:08:42.384Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6756	2024-12-20T23:08:42.384Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6757	2024-12-20T23:08:42.386Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6758	2024-12-20T23:08:42.386Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6759	2024-12-20T23:08:42.387Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6760	2024-12-20T23:08:42.388Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.070162ms iteration = 89 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6761	2024-12-20T23:08:42.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6762	2024-12-20T23:08:42.389Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6763	2024-12-20T23:08:42.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6764	2024-12-20T23:08:42.391Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6765	2024-12-20T23:08:42.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6766	2024-12-20T23:08:42.393Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6767	2024-12-20T23:08:42.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
6768	2024-12-20T23:08:42.396Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6769	2024-12-20T23:08:42.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6770	2024-12-20T23:08:42.405Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
6771	2024-12-20T23:08:42.405Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.222422ms iteration = 89 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
6772	2024-12-20T23:08:43.147Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 262 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 90643bc7-3b54-42b6-bfb9-37bc6042f247 response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
6773	2024-12-20T23:08:52.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 90 reason = Timeout
6774	2024-12-20T23:08:52.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 90 reason = Timeout
6775	2024-12-20T23:08:52.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6776	2024-12-20T23:08:52.385Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6777	2024-12-20T23:08:52.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6778	2024-12-20T23:08:52.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6779	2024-12-20T23:08:52.387Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6780	2024-12-20T23:08:52.387Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6781	2024-12-20T23:08:52.387Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6782	2024-12-20T23:08:52.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6783	2024-12-20T23:08:52.389Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6784	2024-12-20T23:08:52.389Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.467622ms iteration = 90 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6785	2024-12-20T23:08:52.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6786	2024-12-20T23:08:52.390Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6787	2024-12-20T23:08:52.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6788	2024-12-20T23:08:52.392Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6789	2024-12-20T23:08:52.395Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
6790	2024-12-20T23:08:52.395Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6791	2024-12-20T23:08:52.395Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6792	2024-12-20T23:08:52.406Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
6793	2024-12-20T23:08:52.406Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.973306ms iteration = 90 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
6794	2024-12-20T23:08:53.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 244 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 0d373695-70fe-4a6e-be51-1f2341c26f2f response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
6795	2024-12-20T23:08:58.137Z	INFO	nexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:48053
6796	2024-12-20T23:08:58.137Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48053 req_id = 19a1db0e-90c1-46c7-b95e-ce5e8ff4d3db roles = RoleSet { roles: {} } uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?sort_by=id_ascending
6797	2024-12-20T23:08:58.138Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48053 req_id = 19a1db0e-90c1-46c7-b95e-ce5e8ff4d3db resource = Database result = Ok(()) uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?sort_by=id_ascending
6798	2024-12-20T23:08:58.141Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3255 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48053 req_id = 19a1db0e-90c1-46c7-b95e-ce5e8ff4d3db response_code = 200 uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?sort_by=id_ascending
6799	2024-12-20T23:08:58.141Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48053 req_id = 8b440826-64f7-4ab4-a47b-8fdc0ce327fe roles = RoleSet { roles: {} } uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmM2I2MGJkMy02ZTM0LTQxNDAtYTc1NS01N2YyNmRiYzViMzQifX0%3D
6800	2024-12-20T23:08:58.141Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48053 req_id = 8b440826-64f7-4ab4-a47b-8fdc0ce327fe resource = Database result = Ok(()) uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmM2I2MGJkMy02ZTM0LTQxNDAtYTc1NS01N2YyNmRiYzViMzQifX0%3D
6801	2024-12-20T23:08:58.143Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2422 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48053 req_id = 8b440826-64f7-4ab4-a47b-8fdc0ce327fe response_code = 200 uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmM2I2MGJkMy02ZTM0LTQxNDAtYTc1NS01N2YyNmRiYzViMzQifX0%3D
6802	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 31 reason = Timeout
6803	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 31 reason = Timeout
6804	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
6805	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
6806	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6807	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 31 reason = Timeout
6808	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
6809	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 16 reason = Timeout
6810	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
6811	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 7 reason = Timeout
6812	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
6813	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 31 reason = Timeout
6814	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 31 reason = Timeout
6815	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6816	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 33 reason = Timeout
6817	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 31 reason = Timeout
6818	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6819	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6820	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6821	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 16 reason = Timeout
6822	2024-12-20T23:09:02.384Z	INFO	nexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
6823	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
6824	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 31 reason = Timeout
6825	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 18 reason = Timeout
6826	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 16 reason = Timeout
6827	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 16 reason = Timeout
6828	2024-12-20T23:09:02.384Z	INFO	nexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-20 22:59:02.384547369 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
6829	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
6830	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
6831	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 31 reason = Timeout
6832	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
6833	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 31 reason = Timeout
6834	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
6835	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 16 reason = Timeout
6836	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = external_endpoints iteration = 16 reason = Timeout
6837	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 18 reason = Timeout
6838	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6839	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 17 reason = Timeout
6840	2024-12-20T23:09:02.384Z	WARN	nexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7
6841	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 91 reason = Timeout
6842	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 14.58\u{b5}s iteration = 17 status = {"enabled":false,"target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7"}
6843	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 31 reason = Timeout
6844	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6845	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 91 reason = Timeout
6846	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
6847	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6848	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 16 reason = Timeout
6849	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 16 reason = Timeout
6850	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
6851	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
6852	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6853	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6854	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 31 reason = Timeout
6855	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
6856	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 31 reason = Timeout
6857	2024-12-20T23:09:02.384Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
6858	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6859	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
6860	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
6861	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6862	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 31 reason = Timeout
6863	2024-12-20T23:09:02.384Z	DEBG	nexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 16 reason = Timeout
6864	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
6865	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 16 reason = Timeout
6866	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
6867	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 31 reason = Timeout
6868	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 4 reason = Timeout
6869	2024-12-20T23:09:02.385Z	DEBG	nexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
6870	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6871	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6872	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
6873	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
6874	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
6875	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6876	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
6877	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
6878	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6879	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6880	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6881	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
6882	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6883	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
6884	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
6885	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6886	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
6887	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
6888	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
6889	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
6890	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6891	2024-12-20T23:09:02.385Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1e296b8d-6800-4056-8034-a1ec2d005fff.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("ace0592c-7755-46f8-b47a-c9b6444fe64c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("ace0592c-7755-46f8-b47a-c9b6444fe64c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1557, tv_nsec: 775480168 } })
6892	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6893	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
6894	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6895	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6896	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6897	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6898	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6899	2024-12-20T23:09:02.385Z	DEBG	nexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("616ce87c-1f39-49d9-ba35-ee63b2650ecd.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b4615881-2f68-4c82-be17-071bf3ae3641.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c9217e92-0af3-4930-acb2-aa82ded5b782.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("c9217e92-0af3-4930-acb2-aa82ded5b782.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 1557, tv_nsec: 775593701 } })
6900	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
6901	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
6902	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
6903	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
6904	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
6905	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.10442ms iteration = 16 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
6906	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
6907	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
6908	2024-12-20T23:09:02.385Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.138281ms iteration = 16 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
6909	2024-12-20T23:09:02.387Z	DEBG	nexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 2.88742ms iteration = 16 status = {"errors":[],"found_port_ok":[]}
6910	2024-12-20T23:09:02.387Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1557, tv_nsec: 777829502 } })
6911	2024-12-20T23:09:02.387Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.176757ms iteration = 31 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
6912	2024-12-20T23:09:02.387Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
6913	2024-12-20T23:09:02.388Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6914	2024-12-20T23:09:02.389Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6915	2024-12-20T23:09:02.389Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6916	2024-12-20T23:09:02.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
6917	2024-12-20T23:09:02.390Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
6918	2024-12-20T23:09:02.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
6919	2024-12-20T23:09:02.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
6920	2024-12-20T23:09:02.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
6921	2024-12-20T23:09:02.392Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6922	2024-12-20T23:09:02.392Z	DEBG	nexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
6923	2024-12-20T23:09:02.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
6924	2024-12-20T23:09:02.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6925	2024-12-20T23:09:02.395Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6926	2024-12-20T23:09:02.396Z	INFO	nexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-20 22:59:02.384547369 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
6927	2024-12-20T23:09:02.396Z	DEBG	nexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 12.315298ms iteration = 16 status = {"expiration":"2024-12-20T22:59:02.384547369Z","pruned":[]}
6928	2024-12-20T23:09:02.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
6929	2024-12-20T23:09:02.396Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
6930	2024-12-20T23:09:02.397Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1557, tv_nsec: 787193980 } })
6931	2024-12-20T23:09:02.397Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
6932	2024-12-20T23:09:02.397Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
6933	2024-12-20T23:09:02.398Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6934	2024-12-20T23:09:02.399Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6935	2024-12-20T23:09:02.399Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6936	2024-12-20T23:09:02.400Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6937	2024-12-20T23:09:02.401Z	DEBG	nexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
6938	2024-12-20T23:09:02.401Z	INFO	nexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
6939	2024-12-20T23:09:02.401Z	DEBG	nexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 16.192474ms iteration = 16 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
6940	2024-12-20T23:09:02.402Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6941	2024-12-20T23:09:02.403Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6942	2024-12-20T23:09:02.403Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6943	2024-12-20T23:09:02.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6944	2024-12-20T23:09:02.406Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6945	2024-12-20T23:09:02.406Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6946	2024-12-20T23:09:02.409Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6947	2024-12-20T23:09:02.409Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6948	2024-12-20T23:09:02.409Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6949	2024-12-20T23:09:02.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
6950	2024-12-20T23:09:02.412Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
6951	2024-12-20T23:09:02.412Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
6952	2024-12-20T23:09:02.418Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1557, tv_nsec: 808646259 } })
6953	2024-12-20T23:09:02.418Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
6954	2024-12-20T23:09:02.432Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6955	2024-12-20T23:09:02.433Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
6956	2024-12-20T23:09:02.433Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6957	2024-12-20T23:09:02.433Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6958	2024-12-20T23:09:02.434Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
6959	2024-12-20T23:09:02.434Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
6960	2024-12-20T23:09:02.435Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6961	2024-12-20T23:09:02.436Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6962	2024-12-20T23:09:02.437Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
6963	2024-12-20T23:09:02.437Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 52.847571ms iteration = 31 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
6964	2024-12-20T23:09:02.438Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
6965	2024-12-20T23:09:02.439Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6966	2024-12-20T23:09:02.440Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6967	2024-12-20T23:09:02.440Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6968	2024-12-20T23:09:02.440Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
6969	2024-12-20T23:09:02.442Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
6970	2024-12-20T23:09:02.442Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6971	2024-12-20T23:09:02.442Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6972	2024-12-20T23:09:02.443Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6973	2024-12-20T23:09:02.443Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6974	2024-12-20T23:09:02.443Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6975	2024-12-20T23:09:02.446Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
6976	2024-12-20T23:09:02.446Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
6977	2024-12-20T23:09:02.446Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
6978	2024-12-20T23:09:02.451Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6979	2024-12-20T23:09:02.455Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
6980	2024-12-20T23:09:02.455Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6981	2024-12-20T23:09:02.455Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6982	2024-12-20T23:09:02.455Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6983	2024-12-20T23:09:02.456Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6984	2024-12-20T23:09:02.457Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6985	2024-12-20T23:09:02.457Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6986	2024-12-20T23:09:02.457Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6987	2024-12-20T23:09:02.457Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6988	2024-12-20T23:09:02.457Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6989	2024-12-20T23:09:02.458Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 73.385156ms iteration = 31 status = {"errors":[],"finish_invoked_ok":[]}
6990	2024-12-20T23:09:02.461Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 76.778778ms iteration = 31 status = {"errors":[],"garbage_collect_requested":[]}
6991	2024-12-20T23:09:02.461Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
6992	2024-12-20T23:09:02.461Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
6993	2024-12-20T23:09:02.461Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6994	2024-12-20T23:09:02.461Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
6995	2024-12-20T23:09:02.461Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6996	2024-12-20T23:09:02.461Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
6997	2024-12-20T23:09:02.461Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6998	2024-12-20T23:09:02.461Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6999	2024-12-20T23:09:02.462Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7000	2024-12-20T23:09:02.462Z	DEBG	nexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
7001	2024-12-20T23:09:02.462Z	INFO	nexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
7002	2024-12-20T23:09:02.462Z	DEBG	nexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 77.627161ms iteration = 16 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
7003	2024-12-20T23:09:02.462Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
7004	2024-12-20T23:09:02.462Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7005	2024-12-20T23:09:02.462Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7006	2024-12-20T23:09:02.462Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7007	2024-12-20T23:09:02.463Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
7008	2024-12-20T23:09:02.464Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
7009	2024-12-20T23:09:02.464Z	DEBG	nexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
7010	2024-12-20T23:09:02.464Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
7011	2024-12-20T23:09:02.464Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
7012	2024-12-20T23:09:02.464Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7013	2024-12-20T23:09:02.465Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7014	2024-12-20T23:09:02.466Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7015	2024-12-20T23:09:02.466Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
7016	2024-12-20T23:09:02.466Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7017	2024-12-20T23:09:02.466Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7018	2024-12-20T23:09:02.466Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7019	2024-12-20T23:09:02.466Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
7020	2024-12-20T23:09:02.466Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
7021	2024-12-20T23:09:02.467Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7022	2024-12-20T23:09:02.467Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7023	2024-12-20T23:09:02.468Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 84.075208ms iteration = 91 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
7024	2024-12-20T23:09:02.468Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7025	2024-12-20T23:09:02.469Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7026	2024-12-20T23:09:02.469Z	DEBG	nexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 85.401905ms iteration = 7 status = {"nsuccess":5}
7027	2024-12-20T23:09:02.470Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7028	2024-12-20T23:09:02.471Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7029	2024-12-20T23:09:02.471Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7030	2024-12-20T23:09:02.471Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7031	2024-12-20T23:09:02.473Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
7032	2024-12-20T23:09:02.473Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7033	2024-12-20T23:09:02.474Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7034	2024-12-20T23:09:02.475Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
7035	2024-12-20T23:09:02.475Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
7036	2024-12-20T23:09:02.475Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7037	2024-12-20T23:09:02.475Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
7038	2024-12-20T23:09:02.475Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7039	2024-12-20T23:09:02.476Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7040	2024-12-20T23:09:02.476Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
7041	2024-12-20T23:09:02.476Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 92.209342ms iteration = 91 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
7042	2024-12-20T23:09:02.477Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7043	2024-12-20T23:09:02.477Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7044	2024-12-20T23:09:02.479Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
7045	2024-12-20T23:09:02.479Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7046	2024-12-20T23:09:02.479Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7047	2024-12-20T23:09:02.480Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7048	2024-12-20T23:09:02.481Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7049	2024-12-20T23:09:02.483Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7050	2024-12-20T23:09:02.484Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
7051	2024-12-20T23:09:02.484Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
7052	2024-12-20T23:09:02.484Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
7053	2024-12-20T23:09:02.484Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7054	2024-12-20T23:09:02.485Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7055	2024-12-20T23:09:02.486Z	DEBG	nexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
7056	2024-12-20T23:09:02.486Z	DEBG	nexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
7057	2024-12-20T23:09:02.486Z	DEBG	nexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
7058	2024-12-20T23:09:02.486Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7059	2024-12-20T23:09:02.486Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7060	2024-12-20T23:09:02.486Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7061	2024-12-20T23:09:02.487Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
7062	2024-12-20T23:09:02.487Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7063	2024-12-20T23:09:02.487Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7064	2024-12-20T23:09:02.487Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7065	2024-12-20T23:09:02.491Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
7066	2024-12-20T23:09:02.491Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7067	2024-12-20T23:09:02.491Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7068	2024-12-20T23:09:02.491Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7069	2024-12-20T23:09:02.491Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
7070	2024-12-20T23:09:02.491Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7071	2024-12-20T23:09:02.491Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7072	2024-12-20T23:09:02.493Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
7073	2024-12-20T23:09:02.493Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7074	2024-12-20T23:09:02.493Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7075	2024-12-20T23:09:02.494Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7076	2024-12-20T23:09:02.494Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7077	2024-12-20T23:09:02.496Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7078	2024-12-20T23:09:02.497Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7079	2024-12-20T23:09:02.498Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7080	2024-12-20T23:09:02.498Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7081	2024-12-20T23:09:02.499Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 115.244825ms iteration = 31 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
7082	2024-12-20T23:09:02.499Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7083	2024-12-20T23:09:02.500Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7084	2024-12-20T23:09:02.501Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7085	2024-12-20T23:09:02.501Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7086	2024-12-20T23:09:02.502Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7087	2024-12-20T23:09:02.504Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7088	2024-12-20T23:09:02.504Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7089	2024-12-20T23:09:02.504Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 287d6774-d3e6-4b73-8c77-c90534d4a54d, lookup_type: ById(287d6774-d3e6-4b73-8c77-c90534d4a54d) } result = Ok(())
7090	2024-12-20T23:09:02.504Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7091	2024-12-20T23:09:02.504Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7092	2024-12-20T23:09:02.505Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
7093	2024-12-20T23:09:02.505Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
7094	2024-12-20T23:09:02.505Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
7095	2024-12-20T23:09:02.505Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7096	2024-12-20T23:09:02.505Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
7097	2024-12-20T23:09:02.505Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7098	2024-12-20T23:09:02.506Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 122.139143ms iteration = 31 status = 0
7099	2024-12-20T23:09:02.507Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7100	2024-12-20T23:09:02.507Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7101	2024-12-20T23:09:02.508Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7102	2024-12-20T23:09:02.511Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 126.938356ms iteration = 31 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
7103	2024-12-20T23:09:02.512Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
7104	2024-12-20T23:09:02.512Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7105	2024-12-20T23:09:02.513Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7106	2024-12-20T23:09:02.514Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 129.63914ms iteration = 31 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
7107	2024-12-20T23:09:02.515Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7108	2024-12-20T23:09:02.515Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7109	2024-12-20T23:09:02.516Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7110	2024-12-20T23:09:02.516Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7111	2024-12-20T23:09:02.516Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7112	2024-12-20T23:09:02.516Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7113	2024-12-20T23:09:02.517Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7114	2024-12-20T23:09:02.519Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7115	2024-12-20T23:09:02.519Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7116	2024-12-20T23:09:02.519Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
7117	2024-12-20T23:09:02.519Z	DEBG	nexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7118	2024-12-20T23:09:02.519Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7119	2024-12-20T23:09:02.520Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7120	2024-12-20T23:09:02.521Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7121	2024-12-20T23:09:02.523Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7122	2024-12-20T23:09:02.523Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7123	2024-12-20T23:09:02.526Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7124	2024-12-20T23:09:02.526Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7125	2024-12-20T23:09:02.529Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7126	2024-12-20T23:09:02.531Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
7127	2024-12-20T23:09:02.531Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 1e86bf23-6ffd-4a12-93c7-0061d7d17590 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
7128	2024-12-20T23:09:02.532Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7129	2024-12-20T23:09:02.532Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7130	2024-12-20T23:09:02.533Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
7131	2024-12-20T23:09:02.533Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7132	2024-12-20T23:09:02.534Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7133	2024-12-20T23:09:02.535Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7134	2024-12-20T23:09:02.535Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7135	2024-12-20T23:09:02.537Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7136	2024-12-20T23:09:02.537Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7137	2024-12-20T23:09:02.538Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7138	2024-12-20T23:09:02.539Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7139	2024-12-20T23:09:02.541Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
7140	2024-12-20T23:09:02.541Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7141	2024-12-20T23:09:02.541Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7142	2024-12-20T23:09:02.542Z	DEBG	nexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
7143	2024-12-20T23:09:02.542Z	DEBG	nexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 5fa20c62-75c0-4531-be07-51b87d303d18 dns_zone_name = oxide.test found_so_far = 0 version = 2
7144	2024-12-20T23:09:02.542Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7145	2024-12-20T23:09:02.543Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7146	2024-12-20T23:09:02.544Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 1e86bf23-6ffd-4a12-93c7-0061d7d17590 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
7147	2024-12-20T23:09:02.544Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
7148	2024-12-20T23:09:02.544Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-20 22:54:38.956071 UTC dns_group = internal generation = 1
7149	2024-12-20T23:09:02.544Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 159.410007ms iteration = 16 status = {"generation":1}
7150	2024-12-20T23:09:02.544Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7151	2024-12-20T23:09:02.544Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
7152	2024-12-20T23:09:02.544Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
7153	2024-12-20T23:09:02.544Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 160.291711ms iteration = 31 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
7154	2024-12-20T23:09:02.545Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
7155	2024-12-20T23:09:02.545Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7156	2024-12-20T23:09:02.545Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7157	2024-12-20T23:09:02.548Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
7158	2024-12-20T23:09:02.548Z	DEBG	nexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7159	2024-12-20T23:09:02.548Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7160	2024-12-20T23:09:02.550Z	DEBG	nexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 5fa20c62-75c0-4531-be07-51b87d303d18 dns_zone_name = oxide.test found_so_far = 1 version = 2
7161	2024-12-20T23:09:02.550Z	DEBG	nexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
7162	2024-12-20T23:09:02.550Z	DEBG	nexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-20 22:54:39.899407 UTC dns_group = external generation = 2
7163	2024-12-20T23:09:02.550Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 165.602077ms iteration = 16 status = {"generation":2}
7164	2024-12-20T23:09:02.550Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7165	2024-12-20T23:09:02.550Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7166	2024-12-20T23:09:02.552Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7167	2024-12-20T23:09:02.552Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7168	2024-12-20T23:09:02.554Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7169	2024-12-20T23:09:02.554Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7170	2024-12-20T23:09:02.556Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7171	2024-12-20T23:09:02.556Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7172	2024-12-20T23:09:02.558Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7173	2024-12-20T23:09:02.558Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7174	2024-12-20T23:09:02.559Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7175	2024-12-20T23:09:02.559Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7176	2024-12-20T23:09:02.563Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7177	2024-12-20T23:09:02.563Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7178	2024-12-20T23:09:02.566Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7179	2024-12-20T23:09:02.568Z	DEBG	nexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-20T22:53:58.073373Z, time_modified: 2024-12-20T22:53:58.073373Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, db_silo: Silo { identity: SiloIdentity { id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-20T22:54:39.847015Z, time_modified: 2024-12-20T22:54:39.847015Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, db_silo: Silo { identity: SiloIdentity { id: ee579470-b16b-4af1-ad3a-b3d29f9a2fb2, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-20T22:54:39.847015Z, time_modified: 2024-12-20T22:54:39.847015Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156" }] }) }
7180	2024-12-20T23:09:02.568Z	DEBG	nexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 184.128176ms iteration = 16 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"ee579470-b16b-4af1-ad3a-b3d29f9a2fb2","tls_certs":["ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156"]}},"default_endpoint":{"silo_id":"ee579470-b16b-4af1-ad3a-b3d29f9a2fb2","tls_certs":["ddd68eba2fe2b4671931f4479dcf9e4c9f8c5ba6a9fe1416a4f39551a052d156"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
7181	2024-12-20T23:09:02.571Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
7182	2024-12-20T23:09:02.571Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7183	2024-12-20T23:09:02.571Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7184	2024-12-20T23:09:02.583Z	WARN	nexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
7185	2024-12-20T23:09:02.583Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 198.8519ms iteration = 16 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
7186	2024-12-20T23:09:02.587Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7187	2024-12-20T23:09:02.587Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7188	2024-12-20T23:09:02.589Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7189	2024-12-20T23:09:02.590Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7190	2024-12-20T23:09:02.590Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7191	2024-12-20T23:09:02.590Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7192	2024-12-20T23:09:02.591Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 207.289652ms iteration = 33 status = {"physical_disks_added":0}
7193	2024-12-20T23:09:02.592Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 zone_address = fd00:1122:3344:101::2
7194	2024-12-20T23:09:02.592Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7195	2024-12-20T23:09:02.592Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7196	2024-12-20T23:09:02.592Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c88e87fd-2df1-4f35-bb09-bb63b85ccb37", "content-length": "26", "date": "Fri, 20 Dec 2024 23:09:02 GMT"} })
7197	2024-12-20T23:09:02.592Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7198	2024-12-20T23:09:02.593Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
7199	2024-12-20T23:09:02.594Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7200	2024-12-20T23:09:02.594Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7201	2024-12-20T23:09:02.594Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7202	2024-12-20T23:09:02.600Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7203	2024-12-20T23:09:02.600Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7204	2024-12-20T23:09:02.673Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
7205	2024-12-20T23:09:02.690Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
7206	2024-12-20T23:09:02.690Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7207	2024-12-20T23:09:02.690Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d1e99d51-d6dd-4585-b16f-7a0cd171a6b7", "content-length": "26", "date": "Fri, 20 Dec 2024 23:09:02 GMT"} })
7208	2024-12-20T23:09:02.690Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7209	2024-12-20T23:09:02.692Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
7210	2024-12-20T23:09:02.692Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7211	2024-12-20T23:09:02.692Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "64aa1724-a6a7-4565-9009-c92b2b2e4941", "content-length": "26", "date": "Fri, 20 Dec 2024 23:09:02 GMT"} })
7212	2024-12-20T23:09:02.692Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7213	2024-12-20T23:09:02.693Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
7214	2024-12-20T23:09:02.695Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
7215	2024-12-20T23:09:02.697Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
7216	2024-12-20T23:09:02.745Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
7217	2024-12-20T23:09:02.745Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
7218	2024-12-20T23:09:02.745Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ae30e1c-63d0-4698-a4f5-bd68135e3048", "content-length": "372", "date": "Fri, 20 Dec 2024 23:09:02 GMT"} })
7219	2024-12-20T23:09:02.745Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7220	2024-12-20T23:09:02.746Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7221	2024-12-20T23:09:02.747Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7222	2024-12-20T23:09:02.747Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7223	2024-12-20T23:09:02.753Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7224	2024-12-20T23:09:02.758Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
7225	2024-12-20T23:09:02.758Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7226	2024-12-20T23:09:02.758Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7227	2024-12-20T23:09:02.773Z	DEBG	nexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 9b5312da-a277-4244-baaa-33815f0464e4, kind: Service { id: ace0592c-7755-46f8-b47a-c9b6444fe64c }, name: Name("external-dns-ace0592c-7755-46f8-b47a-c9b6444fe64c"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 142, 244])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b6a4ac58-71fd-4271-af9e-04ea5fb7455a, kind: Service { id: 1e296b8d-6800-4056-8034-a1ec2d005fff }, name: Name("external-dns-1e296b8d-6800-4056-8034-a1ec2d005fff"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 157, 244])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 1e21a6ac-6d39-461b-a43a-59a0e5d45557, kind: Service { id: a67567ae-7026-453e-acd2-e1780994c729 }, name: Name("nexus-a67567ae-7026-453e-acd2-e1780994c729"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 188, 28])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 2351e16d-ccc3-42df-b594-9d3a6ec23973, kind: Service { id: f3b60bd3-6e34-4140-a755-57f26dbc5b34 }, name: Name("nexus-f3b60bd3-6e34-4140-a755-57f26dbc5b34"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 142, 57])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8be612fb-34b6-43da-9363-17ad360f283a, kind: Service { id: 371fea12-332e-4334-902e-d2154e69e006 }, name: Name("nexus-371fea12-332e-4334-902e-d2154e69e006"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 100])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
7228	2024-12-20T23:09:02.773Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7229	2024-12-20T23:09:02.774Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7230	2024-12-20T23:09:02.775Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7231	2024-12-20T23:09:02.777Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
7232	2024-12-20T23:09:02.777Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7233	2024-12-20T23:09:02.777Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7234	2024-12-20T23:09:02.780Z	DEBG	nexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
7235	2024-12-20T23:09:02.780Z	DEBG	nexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 9b5312da-a277-4244-baaa-33815f0464e4, kind: Service { id: ace0592c-7755-46f8-b47a-c9b6444fe64c }, name: Name("external-dns-ace0592c-7755-46f8-b47a-c9b6444fe64c"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 142, 244])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b6a4ac58-71fd-4271-af9e-04ea5fb7455a, kind: Service { id: 1e296b8d-6800-4056-8034-a1ec2d005fff }, name: Name("external-dns-1e296b8d-6800-4056-8034-a1ec2d005fff"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 157, 244])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 1e21a6ac-6d39-461b-a43a-59a0e5d45557, kind: Service { id: a67567ae-7026-453e-acd2-e1780994c729 }, name: Name("nexus-a67567ae-7026-453e-acd2-e1780994c729"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 188, 28])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 2351e16d-ccc3-42df-b594-9d3a6ec23973, kind: Service { id: f3b60bd3-6e34-4140-a755-57f26dbc5b34 }, name: Name("nexus-f3b60bd3-6e34-4140-a755-57f26dbc5b34"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 142, 57])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8be612fb-34b6-43da-9363-17ad360f283a, kind: Service { id: 371fea12-332e-4334-902e-d2154e69e006 }, name: Name("nexus-371fea12-332e-4334-902e-d2154e69e006"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 100])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
7236	2024-12-20T23:09:02.780Z	DEBG	nexus (service-firewall-rule-progator): resolved 2 rules for sleds background_task = service_firewall_rule_propagation
7237	2024-12-20T23:09:02.822Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
7238	2024-12-20T23:09:02.822Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
7239	2024-12-20T23:09:02.823Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "49209878-0f3c-4a39-8768-c7f9084bc91c", "date": "Fri, 20 Dec 2024 23:09:02 GMT"} })
7240	2024-12-20T23:09:02.823Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7241	2024-12-20T23:09:02.823Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7242	2024-12-20T23:09:02.830Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7243	2024-12-20T23:09:02.830Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7244	2024-12-20T23:09:02.834Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7245	2024-12-20T23:09:02.834Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7246	2024-12-20T23:09:02.838Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
7247	2024-12-20T23:09:02.838Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
7248	2024-12-20T23:09:02.839Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "44821fc9-33b9-4198-9709-116f53f3e1b2", "date": "Fri, 20 Dec 2024 23:09:02 GMT"} })
7249	2024-12-20T23:09:02.841Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 457.039649ms iteration = 31 status = {}
7250	2024-12-20T23:09:02.847Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 463.288851ms iteration = 31 status = {}
7251	2024-12-20T23:09:02.859Z	DEBG	nexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: 287d6774-d3e6-4b73-8c77-c90534d4a54d, time_created: 2024-12-20T22:54:34.405180Z, time_modified: 2024-12-20T22:54:34.405180Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: b394d0c6-19a8-4136-9148-af6c47c1f8e0, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(51474595840)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::1:0, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)) }]
7252	2024-12-20T23:09:02.859Z	DEBG	nexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
7253	2024-12-20T23:09:02.859Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7254	2024-12-20T23:09:02.859Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7255	2024-12-20T23:09:02.861Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7256	2024-12-20T23:09:02.862Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7257	2024-12-20T23:09:02.862Z	DEBG	nexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7258	2024-12-20T23:09:02.865Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: 287d6774-d3e6-4b73-8c77-c90534d4a54d, lookup_type: ById(287d6774-d3e6-4b73-8c77-c90534d4a54d) } result = Ok(())
7259	2024-12-20T23:09:02.874Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
7260	2024-12-20T23:09:02.875Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "3db756e1-f81f-4161-b931-e292590b7ae5", "content-length": "2", "date": "Fri, 20 Dec 2024 23:09:02 GMT"} })
7261	2024-12-20T23:09:02.877Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7262	2024-12-20T23:09:02.877Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7263	2024-12-20T23:09:02.879Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 494.914429ms iteration = 31 status = null
7264	2024-12-20T23:09:02.976Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
7265	2024-12-20T23:09:02.980Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
7266	2024-12-20T23:09:02.980Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc614b81-c20a-483a-8043-599c3dbcdd5d", "content-length": "691", "date": "Fri, 20 Dec 2024 23:09:02 GMT"} })
7267	2024-12-20T23:09:02.980Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
7268	2024-12-20T23:09:02.980Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
7269	2024-12-20T23:09:02.980Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
7270	2024-12-20T23:09:02.982Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 597.527607ms iteration = 31 status = {}
7271	2024-12-20T23:09:02.995Z	DEBG	nexus (service-firewall-rule-progator): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
7272	2024-12-20T23:09:02.996Z	DEBG	nexus (service-firewall-rule-progator): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "9369df51-e1ed-4836-ba73-882824de7a59", "date": "Fri, 20 Dec 2024 23:09:02 GMT"} })
7273	2024-12-20T23:09:02.997Z	INFO	nexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:496
7274	2024-12-20T23:09:02.997Z	DEBG	nexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 612.53802ms
7275	2024-12-20T23:09:02.997Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 612.54694ms iteration = 4 status = {}
7276	2024-12-20T23:09:02.999Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
7277	2024-12-20T23:09:03.000Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "835e1b81-2306-407d-81f5-0e57cc644859", "date": "Fri, 20 Dec 2024 23:09:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
7278	2024-12-20T23:09:03.000Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
7279	2024-12-20T23:09:03.001Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "f99c6372-9759-43f4-96c4-0a1b8c50c023", "date": "Fri, 20 Dec 2024 23:09:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
7280	2024-12-20T23:09:03.001Z	INFO	nexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
7281	2024-12-20T23:09:03.004Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
7282	2024-12-20T23:09:03.004Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 619.640125ms iteration = 18 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
7283	2024-12-20T23:09:03.021Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7284	2024-12-20T23:09:03.022Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7285	2024-12-20T23:09:03.024Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 03dd12a0-dc2d-46e1-a7f4-d3397b9b7aa7, rack_id: b394d0c6-19a8-4136-9148-af6c47c1f8e0, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(560ca89c-7c0c-4921-8e5c-b57676f695b0) } rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
7286	2024-12-20T23:09:03.024Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7287	2024-12-20T23:09:03.024Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7288	2024-12-20T23:09:03.033Z	DEBG	nexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
7289	2024-12-20T23:09:03.033Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "d6f263fb-d023-4b96-a856-309d43c74f44", "date": "Fri, 20 Dec 2024 23:09:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
7290	2024-12-20T23:09:03.033Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
7291	2024-12-20T23:09:03.035Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "7a4b18ab-14d7-4762-9e70-824ea01a24ff", "date": "Fri, 20 Dec 2024 23:09:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
7292	2024-12-20T23:09:03.035Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
7293	2024-12-20T23:09:03.035Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
7294	2024-12-20T23:09:03.036Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "248d6f97-33b1-4b01-bc65-ce5e55a2f235", "content-length": "134", "date": "Fri, 20 Dec 2024 23:09:03 GMT"} })
7295	2024-12-20T23:09:03.036Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
7296	2024-12-20T23:09:03.036Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 560ca89c-7c0c-4921-8e5c-b57676f695b0
7297	2024-12-20T23:09:03.036Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
7298	2024-12-20T23:09:03.036Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
7299	2024-12-20T23:09:03.036Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
7300	2024-12-20T23:09:03.036Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
7301	2024-12-20T23:09:03.036Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
7302	2024-12-20T23:09:03.036Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
7303	2024-12-20T23:09:03.036Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
7304	2024-12-20T23:09:03.036Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
7305	2024-12-20T23:09:03.037Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
7306	2024-12-20T23:09:03.037Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7307	2024-12-20T23:09:03.037Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7308	2024-12-20T23:09:03.037Z	DEBG	nexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "83bfc38f-530c-4e36-8cf1-fe61dc83d230", "date": "Fri, 20 Dec 2024 23:09:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
7309	2024-12-20T23:09:03.037Z	INFO	nexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
7310	2024-12-20T23:09:03.039Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
7311	2024-12-20T23:09:03.039Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "598bde46-af27-4a97-9d6a-3740aab441d9", "content-length": "2", "date": "Fri, 20 Dec 2024 23:09:03 GMT"} })
7312	2024-12-20T23:09:03.039Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
7313	2024-12-20T23:09:03.039Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7d56b46-5e09-4ce1-9cd5-8876149ba8d6", "content-length": "52", "date": "Fri, 20 Dec 2024 23:09:03 GMT"} })
7314	2024-12-20T23:09:03.039Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
7315	2024-12-20T23:09:03.040Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
7316	2024-12-20T23:09:03.040Z	INFO	nexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
7317	2024-12-20T23:09:03.040Z	DEBG	nexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 655.724805ms iteration = 18 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
7318	2024-12-20T23:09:03.141Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "964b30f1-3db0-4058-9f5e-8cd71f81f74a", "date": "Fri, 20 Dec 2024 23:09:02 GMT"} })
7319	2024-12-20T23:09:03.141Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch = Switch0
7320	2024-12-20T23:09:03.141Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
7321	2024-12-20T23:09:03.142Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "91f11442-e94b-468f-954f-66adad7be787", "content-length": "682", "date": "Fri, 20 Dec 2024 23:09:02 GMT"} })
7322	2024-12-20T23:09:03.142Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7323	2024-12-20T23:09:03.142Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7324	2024-12-20T23:09:03.146Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7325	2024-12-20T23:09:03.146Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7326	2024-12-20T23:09:03.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 253 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 6fe47280-a8f4-4d72-b7b7-b65d134e8e1c response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
7327	2024-12-20T23:09:03.149Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7328	2024-12-20T23:09:03.150Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7329	2024-12-20T23:09:03.152Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7330	2024-12-20T23:09:03.152Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7331	2024-12-20T23:09:03.158Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 774.446215ms iteration = 31 status = {}
7332	2024-12-20T23:09:12.384Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 92 reason = Timeout
7333	2024-12-20T23:09:12.384Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7334	2024-12-20T23:09:12.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 92 reason = Timeout
7335	2024-12-20T23:09:12.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7336	2024-12-20T23:09:12.384Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7337	2024-12-20T23:09:12.384Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7338	2024-12-20T23:09:12.386Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7339	2024-12-20T23:09:12.388Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7340	2024-12-20T23:09:12.388Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7341	2024-12-20T23:09:12.388Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7342	2024-12-20T23:09:12.390Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7343	2024-12-20T23:09:12.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7344	2024-12-20T23:09:12.392Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7345	2024-12-20T23:09:12.394Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
7346	2024-12-20T23:09:12.394Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7347	2024-12-20T23:09:12.395Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7348	2024-12-20T23:09:12.407Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
7349	2024-12-20T23:09:12.407Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.097174ms iteration = 92 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
7350	2024-12-20T23:09:12.415Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7351	2024-12-20T23:09:12.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7352	2024-12-20T23:09:12.442Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 57.912331ms iteration = 92 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
7353	2024-12-20T23:09:13.138Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48053 req_id = 7a946d8a-bef3-42ff-9e9f-66c8b8e9e2d8 roles = RoleSet { roles: {} } uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?sort_by=id_ascending
7354	2024-12-20T23:09:13.138Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48053 req_id = 7a946d8a-bef3-42ff-9e9f-66c8b8e9e2d8 resource = Database result = Ok(()) uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?sort_by=id_ascending
7355	2024-12-20T23:09:13.141Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3275 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48053 req_id = 7a946d8a-bef3-42ff-9e9f-66c8b8e9e2d8 response_code = 200 uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?sort_by=id_ascending
7356	2024-12-20T23:09:13.141Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48053 req_id = 8e958870-1265-4ef1-8d82-2620d860ff62 roles = RoleSet { roles: {} } uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmM2I2MGJkMy02ZTM0LTQxNDAtYTc1NS01N2YyNmRiYzViMzQifX0%3D
7357	2024-12-20T23:09:13.141Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48053 req_id = 8e958870-1265-4ef1-8d82-2620d860ff62 resource = Database result = Ok(()) uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmM2I2MGJkMy02ZTM0LTQxNDAtYTc1NS01N2YyNmRiYzViMzQifX0%3D
7358	2024-12-20T23:09:13.144Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2581 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48053 req_id = 8e958870-1265-4ef1-8d82-2620d860ff62 response_code = 200 uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmM2I2MGJkMy02ZTM0LTQxNDAtYTc1NS01N2YyNmRiYzViMzQifX0%3D
7359	2024-12-20T23:09:13.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 295 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 89069c1b-b228-49bd-a378-a54432dbc636 response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
7360	2024-12-20T23:09:22.384Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 93 reason = Timeout
7361	2024-12-20T23:09:22.384Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7362	2024-12-20T23:09:22.384Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 93 reason = Timeout
7363	2024-12-20T23:09:22.384Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7364	2024-12-20T23:09:22.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7365	2024-12-20T23:09:22.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7366	2024-12-20T23:09:22.387Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7367	2024-12-20T23:09:22.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7368	2024-12-20T23:09:22.390Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7369	2024-12-20T23:09:22.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7370	2024-12-20T23:09:22.392Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7371	2024-12-20T23:09:22.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7372	2024-12-20T23:09:22.394Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7373	2024-12-20T23:09:22.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
7374	2024-12-20T23:09:22.396Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7375	2024-12-20T23:09:22.397Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7376	2024-12-20T23:09:22.407Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
7377	2024-12-20T23:09:22.407Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.93181ms iteration = 93 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
7378	2024-12-20T23:09:22.419Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7379	2024-12-20T23:09:22.419Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7380	2024-12-20T23:09:22.449Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 64.395978ms iteration = 93 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
7381	2024-12-20T23:09:23.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 283 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = fb3c81b5-9229-4394-88b5-d756886f9a11 response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
7382	2024-12-20T23:09:28.138Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48053 req_id = 6b3d981b-4cf3-467f-9827-d39fa6aaf5f4 roles = RoleSet { roles: {} } uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?sort_by=id_ascending
7383	2024-12-20T23:09:28.139Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48053 req_id = 6b3d981b-4cf3-467f-9827-d39fa6aaf5f4 resource = Database result = Ok(()) uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?sort_by=id_ascending
7384	2024-12-20T23:09:28.141Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2919 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48053 req_id = 6b3d981b-4cf3-467f-9827-d39fa6aaf5f4 response_code = 200 uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?sort_by=id_ascending
7385	2024-12-20T23:09:28.141Z	DEBG	nexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48053 req_id = 1aacd583-215e-4143-be14-e38894142761 roles = RoleSet { roles: {} } uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmM2I2MGJkMy02ZTM0LTQxNDAtYTc1NS01N2YyNmRiYzViMzQifX0%3D
7386	2024-12-20T23:09:28.142Z	DEBG	nexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48053 req_id = 1aacd583-215e-4143-be14-e38894142761 resource = Database result = Ok(()) uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmM2I2MGJkMy02ZTM0LTQxNDAtYTc1NS01N2YyNmRiYzViMzQifX0%3D
7387	2024-12-20T23:09:28.144Z	INFO	nexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2829 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48053 req_id = 1aacd583-215e-4143-be14-e38894142761 response_code = 200 uri = /metrics/collectors/103fc383-10f6-4327-a1b1-dabac4e1e365/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmM2I2MGJkMy02ZTM0LTQxNDAtYTc1NS01N2YyNmRiYzViMzQifX0%3D
7388	2024-12-20T23:09:32.384Z	DEBG	nexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 32 reason = Timeout
7389	2024-12-20T23:09:32.384Z	INFO	nexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
7390	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 32 reason = Timeout
7391	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7392	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
7393	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 94 reason = Timeout
7394	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 34 reason = Timeout
7395	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7396	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7397	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement iteration = 32 reason = Timeout
7398	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
7399	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 94 reason = Timeout
7400	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7401	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_watcher iteration = 32 reason = Timeout
7402	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 32 reason = Timeout
7403	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
7404	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
7405	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = v2p_manager iteration = 32 reason = Timeout
7406	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7407	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 32 reason = Timeout
7408	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 32 reason = Timeout
7409	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7410	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
7411	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = phantom_disks iteration = 32 reason = Timeout
7412	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = bfd_manager iteration = 32 reason = Timeout
7413	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
7414	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 32 reason = Timeout
7415	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7416	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 32 reason = Timeout
7417	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = instance_updater iteration = 32 reason = Timeout
7418	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
7419	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
7420	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7421	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 32 reason = Timeout
7422	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
7423	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7424	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
7425	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
7426	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
7427	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7428	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7429	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
7430	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7431	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
7432	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
7433	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7434	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
7435	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7436	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7437	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7438	2024-12-20T23:09:32.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
7439	2024-12-20T23:09:32.387Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7440	2024-12-20T23:09:32.387Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7441	2024-12-20T23:09:32.388Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7442	2024-12-20T23:09:32.389Z	DEBG	nexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.484813ms iteration = 32 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
7443	2024-12-20T23:09:32.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
7444	2024-12-20T23:09:32.390Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
7445	2024-12-20T23:09:32.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7446	2024-12-20T23:09:32.390Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7447	2024-12-20T23:09:32.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
7448	2024-12-20T23:09:32.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7449	2024-12-20T23:09:32.391Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
7450	2024-12-20T23:09:32.391Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7451	2024-12-20T23:09:32.391Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7452	2024-12-20T23:09:32.392Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7453	2024-12-20T23:09:32.393Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7454	2024-12-20T23:09:32.394Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1587, tv_nsec: 784273520 } })
7455	2024-12-20T23:09:32.394Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
7456	2024-12-20T23:09:32.394Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7457	2024-12-20T23:09:32.394Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 9.812669ms iteration = 32 status = {"errors":[],"finish_invoked_ok":[]}
7458	2024-12-20T23:09:32.395Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7459	2024-12-20T23:09:32.396Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1587, tv_nsec: 786626834 } })
7460	2024-12-20T23:09:32.396Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
7461	2024-12-20T23:09:32.397Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
7462	2024-12-20T23:09:32.397Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7463	2024-12-20T23:09:32.397Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7464	2024-12-20T23:09:32.398Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
7465	2024-12-20T23:09:32.398Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7466	2024-12-20T23:09:32.398Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7467	2024-12-20T23:09:32.401Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7468	2024-12-20T23:09:32.402Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7469	2024-12-20T23:09:32.402Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7470	2024-12-20T23:09:32.402Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7471	2024-12-20T23:09:32.403Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
7472	2024-12-20T23:09:32.403Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7473	2024-12-20T23:09:32.403Z	DEBG	nexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-287d6774-d3e6-4b73-8c77-c90534d4a54d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1587, tv_nsec: 793944805 } })
7474	2024-12-20T23:09:32.403Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
7475	2024-12-20T23:09:32.403Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7476	2024-12-20T23:09:32.404Z	INFO	nexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
7477	2024-12-20T23:09:32.436Z	INFO	nexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
7478	2024-12-20T23:09:32.436Z	DEBG	nexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
7479	2024-12-20T23:09:32.436Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
7480	2024-12-20T23:09:32.437Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
7481	2024-12-20T23:09:32.437Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 52.572513ms iteration = 32 status = {"errors":[],"garbage_collect_requested":[]}
7482	2024-12-20T23:09:32.438Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
7483	2024-12-20T23:09:32.438Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
7484	2024-12-20T23:09:32.440Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
7485	2024-12-20T23:09:32.440Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 55.783252ms iteration = 32 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
7486	2024-12-20T23:09:32.441Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
7487	2024-12-20T23:09:32.443Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 58.226859ms iteration = 94 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
7488	2024-12-20T23:09:32.443Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7489	2024-12-20T23:09:32.443Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 58.457165ms iteration = 32 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
7490	2024-12-20T23:09:32.443Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7491	2024-12-20T23:09:32.443Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7492	2024-12-20T23:09:32.445Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
7493	2024-12-20T23:09:32.445Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7494	2024-12-20T23:09:32.445Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7495	2024-12-20T23:09:32.446Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7496	2024-12-20T23:09:32.446Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7497	2024-12-20T23:09:32.446Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7498	2024-12-20T23:09:32.446Z	INFO	nexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
7499	2024-12-20T23:09:32.446Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 61.611892ms iteration = 32 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
7500	2024-12-20T23:09:32.447Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7501	2024-12-20T23:09:32.448Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
7502	2024-12-20T23:09:32.448Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7503	2024-12-20T23:09:32.448Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7504	2024-12-20T23:09:32.448Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
7505	2024-12-20T23:09:32.449Z	DEBG	nexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7506	2024-12-20T23:09:32.449Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7507	2024-12-20T23:09:32.452Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7508	2024-12-20T23:09:32.454Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
7509	2024-12-20T23:09:32.454Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7510	2024-12-20T23:09:32.455Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7511	2024-12-20T23:09:32.465Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
7512	2024-12-20T23:09:32.465Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 80.991093ms iteration = 94 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
7513	2024-12-20T23:09:32.470Z	INFO	nexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
7514	2024-12-20T23:09:32.470Z	DEBG	nexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
7515	2024-12-20T23:09:32.470Z	DEBG	nexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
7516	2024-12-20T23:09:32.470Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
7517	2024-12-20T23:09:32.472Z	DEBG	nexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
7518	2024-12-20T23:09:32.473Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
7519	2024-12-20T23:09:32.480Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7520	2024-12-20T23:09:32.483Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 287d6774-d3e6-4b73-8c77-c90534d4a54d, lookup_type: ById(287d6774-d3e6-4b73-8c77-c90534d4a54d) } result = Ok(())
7521	2024-12-20T23:09:32.483Z	DEBG	nexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7522	2024-12-20T23:09:32.483Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7523	2024-12-20T23:09:32.486Z	DEBG	nexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 101.832836ms iteration = 32 status = 0
7524	2024-12-20T23:09:32.488Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
7525	2024-12-20T23:09:32.488Z	DEBG	nexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7526	2024-12-20T23:09:32.489Z	DEBG	nexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "17d887c9-6da4-469c-9ade-63258167748f", "content-length": "26", "date": "Fri, 20 Dec 2024 23:09:32 GMT"} })
7527	2024-12-20T23:09:32.489Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7528	2024-12-20T23:09:32.491Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
7529	2024-12-20T23:09:32.492Z	DEBG	nexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 107.601714ms iteration = 34 status = {"physical_disks_added":0}
7530	2024-12-20T23:09:32.503Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 118.939066ms iteration = 32 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
7531	2024-12-20T23:09:32.510Z	INFO	nexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
7532	2024-12-20T23:09:32.510Z	INFO	nexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
7533	2024-12-20T23:09:32.510Z	DEBG	nexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 125.743543ms iteration = 32 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
7534	2024-12-20T23:09:32.518Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7535	2024-12-20T23:09:32.518Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7536	2024-12-20T23:09:32.596Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
7537	2024-12-20T23:09:32.596Z	DEBG	nexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7538	2024-12-20T23:09:32.596Z	DEBG	nexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e9bb288-1a2c-428c-b4ec-cd467fb5e73a", "content-length": "26", "date": "Fri, 20 Dec 2024 23:09:32 GMT"} })
7539	2024-12-20T23:09:32.596Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7540	2024-12-20T23:09:32.597Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
7541	2024-12-20T23:09:32.603Z	INFO	nexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 zone_address = fd00:1122:3344:101::2
7542	2024-12-20T23:09:32.604Z	DEBG	nexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7543	2024-12-20T23:09:32.604Z	DEBG	nexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3a2ae89b-45d0-47af-a630-cc300bcee0bc", "content-length": "26", "date": "Fri, 20 Dec 2024 23:09:32 GMT"} })
7544	2024-12-20T23:09:32.604Z	INFO	nexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7545	2024-12-20T23:09:32.606Z	INFO	nexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
7546	2024-12-20T23:09:32.709Z	DEBG	nexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 324.450289ms iteration = 32 status = {}
7547	2024-12-20T23:09:32.713Z	DEBG	nexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
7548	2024-12-20T23:09:32.713Z	DEBG	nexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e2960bc-0f3f-4de7-bf66-59d26292e79e", "content-length": "2", "date": "Fri, 20 Dec 2024 23:09:32 GMT"} })
7549	2024-12-20T23:09:32.714Z	DEBG	nexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7550	2024-12-20T23:09:32.715Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7551	2024-12-20T23:09:32.718Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
7552	2024-12-20T23:09:32.718Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
7553	2024-12-20T23:09:32.719Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0ed7b08-ea97-4986-b9f7-265c3c816289", "content-length": "372", "date": "Fri, 20 Dec 2024 23:09:32 GMT"} })
7554	2024-12-20T23:09:32.719Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7555	2024-12-20T23:09:32.719Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7556	2024-12-20T23:09:32.721Z	DEBG	nexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 336.14217ms iteration = 32 status = null
7557	2024-12-20T23:09:32.764Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7558	2024-12-20T23:09:32.764Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7559	2024-12-20T23:09:32.766Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7560	2024-12-20T23:09:32.767Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7561	2024-12-20T23:09:32.767Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7562	2024-12-20T23:09:32.767Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7563	2024-12-20T23:09:32.769Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7564	2024-12-20T23:09:32.772Z	DEBG	nexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7565	2024-12-20T23:09:32.772Z	DEBG	nexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7566	2024-12-20T23:09:32.772Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7567	2024-12-20T23:09:32.777Z	INFO	nexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
7568	2024-12-20T23:09:32.777Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
7569	2024-12-20T23:09:32.778Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "2bf221f5-81eb-4f75-8f14-f3f5dfe364e3", "date": "Fri, 20 Dec 2024 23:09:32 GMT"} })
7570	2024-12-20T23:09:32.778Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7571	2024-12-20T23:09:32.778Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7572	2024-12-20T23:09:32.781Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7573	2024-12-20T23:09:32.781Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7574	2024-12-20T23:09:32.784Z	DEBG	nexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7575	2024-12-20T23:09:32.784Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7576	2024-12-20T23:09:32.787Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7577	2024-12-20T23:09:32.788Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7578	2024-12-20T23:09:32.788Z	INFO	nexus (BackgroundTasks): VPC route manager sled 287d6774-d3e6-4b73-8c77-c90534d4a54d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
7579	2024-12-20T23:09:32.788Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
7580	2024-12-20T23:09:32.788Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "60d0e1a1-a559-4977-96b5-899beb4dc805", "date": "Fri, 20 Dec 2024 23:09:32 GMT"} })
7581	2024-12-20T23:09:32.790Z	INFO	nexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 03dd12a0-dc2d-46e1-a7f4-d3397b9b7aa7, rack_id: b394d0c6-19a8-4136-9148-af6c47c1f8e0, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(560ca89c-7c0c-4921-8e5c-b57676f695b0) } rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
7582	2024-12-20T23:09:32.790Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7583	2024-12-20T23:09:32.790Z	DEBG	nexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 405.276538ms iteration = 32 status = {}
7584	2024-12-20T23:09:32.790Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7585	2024-12-20T23:09:32.799Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
7586	2024-12-20T23:09:32.800Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "626ff41c-224f-4148-8f9c-0c8071be5c84", "content-length": "134", "date": "Fri, 20 Dec 2024 23:09:32 GMT"} })
7587	2024-12-20T23:09:32.800Z	INFO	nexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
7588	2024-12-20T23:09:32.800Z	INFO	nexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 560ca89c-7c0c-4921-8e5c-b57676f695b0
7589	2024-12-20T23:09:32.800Z	INFO	nexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
7590	2024-12-20T23:09:32.800Z	INFO	nexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
7591	2024-12-20T23:09:32.800Z	INFO	nexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
7592	2024-12-20T23:09:32.800Z	INFO	nexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
7593	2024-12-20T23:09:32.800Z	INFO	nexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
7594	2024-12-20T23:09:32.800Z	INFO	nexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
7595	2024-12-20T23:09:32.800Z	INFO	nexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
7596	2024-12-20T23:09:32.800Z	INFO	nexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
7597	2024-12-20T23:09:32.800Z	INFO	nexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0
7598	2024-12-20T23:09:32.800Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7599	2024-12-20T23:09:32.801Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7600	2024-12-20T23:09:32.803Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
7601	2024-12-20T23:09:32.803Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e23966a-24dd-48e6-8cc3-1718509279ad", "content-length": "2", "date": "Fri, 20 Dec 2024 23:09:32 GMT"} })
7602	2024-12-20T23:09:32.803Z	DEBG	nexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
7603	2024-12-20T23:09:32.803Z	DEBG	nexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "58d7b43a-e0f1-4950-897c-ded40d06f153", "content-length": "52", "date": "Fri, 20 Dec 2024 23:09:32 GMT"} })
7604	2024-12-20T23:09:32.803Z	INFO	nexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch_location = Switch0
7605	2024-12-20T23:09:32.803Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
7606	2024-12-20T23:09:32.861Z	DEBG	nexus (BackgroundTasks): client request SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
7607	2024-12-20T23:09:32.906Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "03355347-6323-4d49-a2d6-cfda20f7c904", "date": "Fri, 20 Dec 2024 23:09:32 GMT"} })
7608	2024-12-20T23:09:32.906Z	INFO	nexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 switch = Switch0
7609	2024-12-20T23:09:32.906Z	DEBG	nexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
7610	2024-12-20T23:09:32.906Z	DEBG	nexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b394d0c6-19a8-4136-9148-af6c47c1f8e0 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd684e1c-3dcc-4480-9867-dafafce6150d", "content-length": "682", "date": "Fri, 20 Dec 2024 23:09:32 GMT"} })
7611	2024-12-20T23:09:32.906Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7612	2024-12-20T23:09:32.906Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7613	2024-12-20T23:09:32.907Z	DEBG	nexus (BackgroundTasks): client response SledAgent = 287d6774-d3e6-4b73-8c77-c90534d4a54d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "f87261d4-0466-42ed-a29f-ed62ccb463b6", "content-length": "691", "date": "Fri, 20 Dec 2024 23:09:32 GMT"} })
7614	2024-12-20T23:09:32.907Z	INFO	nexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
7615	2024-12-20T23:09:32.907Z	INFO	nexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
7616	2024-12-20T23:09:32.907Z	INFO	nexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
7617	2024-12-20T23:09:32.908Z	DEBG	nexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 523.185226ms iteration = 32 status = {}
7618	2024-12-20T23:09:32.910Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7619	2024-12-20T23:09:32.910Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7620	2024-12-20T23:09:32.913Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7621	2024-12-20T23:09:32.913Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7622	2024-12-20T23:09:32.915Z	DEBG	nexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7623	2024-12-20T23:09:32.915Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7624	2024-12-20T23:09:32.922Z	DEBG	nexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 537.339924ms iteration = 32 status = {}
7625	2024-12-20T23:09:33.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 259 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = caefd5bd-d4eb-46ff-8f7c-868a08a158f2 response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34
7626	2024-12-20T23:09:42.384Z	DEBG	nexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 95 reason = Timeout
7627	2024-12-20T23:09:42.385Z	DEBG	nexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 95 reason = Timeout
7628	2024-12-20T23:09:42.385Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7629	2024-12-20T23:09:42.385Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7630	2024-12-20T23:09:42.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7631	2024-12-20T23:09:42.385Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7632	2024-12-20T23:09:42.387Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7633	2024-12-20T23:09:42.389Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7634	2024-12-20T23:09:42.389Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7635	2024-12-20T23:09:42.390Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7636	2024-12-20T23:09:42.392Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7637	2024-12-20T23:09:42.392Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7638	2024-12-20T23:09:42.393Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7639	2024-12-20T23:09:42.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 83600bd4-f94b-4925-a0e3-cb9a140dfee7, lookup_type: ById(83600bd4-f94b-4925-a0e3-cb9a140dfee7) } result = Ok(())
7640	2024-12-20T23:09:42.396Z	DEBG	nexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7641	2024-12-20T23:09:42.396Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7642	2024-12-20T23:09:42.405Z	DEBG	nexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 original_time_created = 2024-12-20 22:53:54.070733 UTC target_id = 83600bd4-f94b-4925-a0e3-cb9a140dfee7 time_created = 2024-12-20 22:53:54.070733 UTC
7643	2024-12-20T23:09:42.405Z	DEBG	nexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.527703ms iteration = 95 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"83600bd4-f94b-4925-a0e3-cb9a140dfee7","time_created":"2024-12-20T22:53:54.070733Z"}
7644	2024-12-20T23:09:42.414Z	DEBG	nexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7645	2024-12-20T23:09:42.415Z	DEBG	nexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7646	2024-12-20T23:09:42.416Z	DEBG	nexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.472844ms iteration = 95 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
7647	2024-12-20T23:09:43.148Z	INFO	nexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 294 local_addr = [fd00:1122:3344:101::c]:44954 method = GET remote_addr = [fd00:1122:3344:101::d]:54347 req_id = 3c265f3b-5246-4513-849b-8ed16d1d21bf response_code = 200 uri = /f3b60bd3-6e34-4140-a755-57f26dbc5b34