Max level shown:
12025-09-03T04:15:07.349ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 13 reason = Timeout
22025-09-03T04:15:07.349ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
32025-09-03T04:15:07.350ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:133 reason = blueprint disabled target_id = 60e73e54-9659-486a-96ba-b69df28d8796
42025-09-03T04:15:07.350ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 37.72\u{b5}s iteration = 13 status = {"enabled":false,"target_id":"60e73e54-9659-486a-96ba-b69df28d8796"}
52025-09-03T04:15:07.350ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 12 reason = Timeout
62025-09-03T04:15:07.350ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 14 reason = Timeout
72025-09-03T04:15:07.350ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
82025-09-03T04:15:07.350ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 23 reason = Timeout
92025-09-03T04:15:07.350ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
102025-09-03T04:15:07.350ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 14 reason = Timeout
112025-09-03T04:15:07.350ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 23 reason = Timeout
122025-09-03T04:15:07.350ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 23 reason = Timeout
132025-09-03T04:15:07.350ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
142025-09-03T04:15:07.350ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 12 reason = Timeout
152025-09-03T04:15:07.350ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-09-03 04:05:07.349982565 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
162025-09-03T04:15:07.350ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
172025-09-03T04:15:07.350ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 12 reason = Timeout
182025-09-03T04:15:07.350ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 12 reason = Timeout
192025-09-03T04:15:07.350ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 12 reason = Timeout
202025-09-03T04:15:07.350ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
212025-09-03T04:15:07.350ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
222025-09-03T04:15:07.350ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 12 reason = Timeout
232025-09-03T04:15:07.350ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
242025-09-03T04:15:07.350ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
252025-09-03T04:15:07.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
262025-09-03T04:15:07.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
272025-09-03T04:15:07.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
282025-09-03T04:15:07.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
292025-09-03T04:15:07.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
302025-09-03T04:15:07.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
312025-09-03T04:15:07.350ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2085efc3-987d-4d25-bf4e-dbfa977a0d96.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a853d036-9d99-4741-85fc-3823419b4ac1.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("a853d036-9d99-4741-85fc-3823419b4ac1.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1719, tv_nsec: 24303722 } })
322025-09-03T04:15:07.350ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0d788cd6-59e1-4eee-8401-73a64a0795db.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("33692814-cb11-4468-bf51-4d6d8418d856.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("808b9b46-4a4d-447d-99d9-09db72c60cdc.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("808b9b46-4a4d-447d-99d9-09db72c60cdc.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 1719, tv_nsec: 24491433 } })
332025-09-03T04:15:07.351ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
342025-09-03T04:15:07.351ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.270919ms iteration = 12 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
352025-09-03T04:15:07.351ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
362025-09-03T04:15:07.351ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.522191ms iteration = 12 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353"]}
372025-09-03T04:15:07.351ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
382025-09-03T04:15:07.352ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
392025-09-03T04:15:07.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
402025-09-03T04:15:07.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
412025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 15 reason = Timeout
422025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
432025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 23 reason = Timeout
442025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 15.97\u{b5}s iteration = 15 status = "Disabled"
452025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
462025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 23 reason = Timeout
472025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 23 reason = Timeout
482025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
492025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
502025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 67 reason = Timeout
512025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
522025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 23 reason = Timeout
532025-09-03T04:15:07.353ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
542025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
552025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 23 reason = Timeout
562025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
572025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 133 reason = Timeout
582025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
592025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 67 reason = Timeout
602025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 12 reason = Timeout
612025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
622025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
632025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 23 reason = Timeout
642025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
652025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 12 reason = Timeout
662025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
672025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 12 reason = Timeout
682025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
692025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 23 reason = Timeout
702025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
712025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 12 reason = Timeout
722025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
732025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 12 reason = Timeout
742025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
752025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 12 reason = Timeout
762025-09-03T04:15:07.353ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
772025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
782025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 23 reason = Timeout
792025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
802025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 23 reason = Timeout
812025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 23 reason = Timeout
822025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 22.88\u{b5}s iteration = 23 status = {"disabled":true,"errors":[],"sps":[]}
832025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
842025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 25 reason = Timeout
852025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 23 reason = Timeout
862025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
872025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
882025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
892025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 23 reason = Timeout
902025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
912025-09-03T04:15:07.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
922025-09-03T04:15:07.354ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 23 reason = Timeout
932025-09-03T04:15:07.354ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
942025-09-03T04:15:07.354ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
952025-09-03T04:15:07.354ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 23 reason = Timeout
962025-09-03T04:15:07.354ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
972025-09-03T04:15:07.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
982025-09-03T04:15:07.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
992025-09-03T04:15:07.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
1002025-09-03T04:15:07.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
1012025-09-03T04:15:07.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
1022025-09-03T04:15:07.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
1032025-09-03T04:15:07.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1042025-09-03T04:15:07.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
1052025-09-03T04:15:07.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
1062025-09-03T04:15:07.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
1072025-09-03T04:15:07.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
1082025-09-03T04:15:07.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
1092025-09-03T04:15:07.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
1102025-09-03T04:15:07.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
1112025-09-03T04:15:07.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1122025-09-03T04:15:07.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
1132025-09-03T04:15:07.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
1142025-09-03T04:15:07.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1152025-09-03T04:15:07.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
1162025-09-03T04:15:07.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
1172025-09-03T04:15:07.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
1182025-09-03T04:15:07.354ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1719, tv_nsec: 28381352 } })
1192025-09-03T04:15:07.355ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1719, tv_nsec: 28639594 } })
1202025-09-03T04:15:07.355ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1206
1212025-09-03T04:15:07.355ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1206
1222025-09-03T04:15:07.358ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1232025-09-03T04:15:07.358ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1242025-09-03T04:15:07.358ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1252025-09-03T04:15:07.358ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
1262025-09-03T04:15:07.358ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1272025-09-03T04:15:07.358ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1282025-09-03T04:15:07.358ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1292025-09-03T04:15:07.358ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1302025-09-03T04:15:07.358ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
1312025-09-03T04:15:07.358ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1322025-09-03T04:15:07.359ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-09-03 04:05:07.349982565 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
1332025-09-03T04:15:07.359ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 9.489529ms iteration = 12 status = {"expiration":"2025-09-03T04:05:07.349982565Z","pruned":[]}
1342025-09-03T04:15:07.359ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 6.195254ms iteration = 12 status = {"errors":[],"found_port_ok":[]}
1352025-09-03T04:15:07.361ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1362025-09-03T04:15:07.361ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1372025-09-03T04:15:07.363ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1382025-09-03T04:15:07.364ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
1392025-09-03T04:15:07.364ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 10.728228ms iteration = 12 status = {"by_rx":{},"error":null}
1402025-09-03T04:15:07.365ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
1412025-09-03T04:15:07.365ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
1422025-09-03T04:15:07.365ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
1432025-09-03T04:15:07.365ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1442025-09-03T04:15:07.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
1452025-09-03T04:15:07.365ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
1462025-09-03T04:15:07.365ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1472025-09-03T04:15:07.365ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 11.759275ms iteration = 23 status = {"errors":[],"finish_invoked_ok":[]}
1482025-09-03T04:15:07.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1492025-09-03T04:15:07.365ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 15.927255ms iteration = 23 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
1502025-09-03T04:15:07.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1512025-09-03T04:15:07.366ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
1522025-09-03T04:15:07.366ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
1532025-09-03T04:15:07.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
1542025-09-03T04:15:07.366ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1552025-09-03T04:15:07.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1562025-09-03T04:15:07.367ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
1572025-09-03T04:15:07.367ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1582025-09-03T04:15:07.367ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1592025-09-03T04:15:07.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1602025-09-03T04:15:07.367ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1612025-09-03T04:15:07.367ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1622025-09-03T04:15:07.367ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1632025-09-03T04:15:07.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
1642025-09-03T04:15:07.368ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
1652025-09-03T04:15:07.368ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
1662025-09-03T04:15:07.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
1672025-09-03T04:15:07.368ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
1682025-09-03T04:15:07.368ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1692025-09-03T04:15:07.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1702025-09-03T04:15:07.369ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1712025-09-03T04:15:07.369ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1722025-09-03T04:15:07.369ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
1732025-09-03T04:15:07.369ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
1742025-09-03T04:15:07.369ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
1752025-09-03T04:15:07.369ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
1762025-09-03T04:15:07.369ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
1772025-09-03T04:15:07.369ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1782025-09-03T04:15:07.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1792025-09-03T04:15:07.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
1802025-09-03T04:15:07.369ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 16.173466ms iteration = 67 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
1812025-09-03T04:15:07.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
1822025-09-03T04:15:07.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1832025-09-03T04:15:07.369ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
1842025-09-03T04:15:07.370ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
1852025-09-03T04:15:07.370ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
1862025-09-03T04:15:07.370ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:138 found = 0 not_ready_to_be_deleted = 0
1872025-09-03T04:15:07.370ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 16.318059ms iteration = 12 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
1882025-09-03T04:15:07.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
1892025-09-03T04:15:07.370ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
1902025-09-03T04:15:07.370ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
1912025-09-03T04:15:07.370ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
1922025-09-03T04:15:07.370ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1932025-09-03T04:15:07.370ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1942025-09-03T04:15:07.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1952025-09-03T04:15:07.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
1962025-09-03T04:15:07.371ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1972025-09-03T04:15:07.371ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1982025-09-03T04:15:07.372ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1992025-09-03T04:15:07.373ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 19.31486ms iteration = 12 status = {"dispatched":[],"errors":[],"glob_version":"186.0.0","globs_reprocessed":{},"no_receivers":[]}
2002025-09-03T04:15:07.373ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2012025-09-03T04:15:07.374ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2022025-09-03T04:15:07.378ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
2032025-09-03T04:15:07.378ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2042025-09-03T04:15:07.378ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2052025-09-03T04:15:07.378ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2062025-09-03T04:15:07.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2072025-09-03T04:15:07.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2082025-09-03T04:15:07.378ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2092025-09-03T04:15:07.378ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2102025-09-03T04:15:07.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2112025-09-03T04:15:07.380ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 60e73e54-9659-486a-96ba-b69df28d8796, lookup_type: ById(60e73e54-9659-486a-96ba-b69df28d8796) } result = Ok(())
2122025-09-03T04:15:07.380ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2132025-09-03T04:15:07.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
2142025-09-03T04:15:07.381ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1719, tv_nsec: 55110535 } })
2152025-09-03T04:15:07.382ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
2162025-09-03T04:15:07.382ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 28.852849ms iteration = 133 status = {"chicken_switches_updated":false}
2172025-09-03T04:15:07.382ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2182025-09-03T04:15:07.383ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1206 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40
2192025-09-03T04:15:07.386ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2202025-09-03T04:15:07.386ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
2212025-09-03T04:15:07.386ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
2222025-09-03T04:15:07.387ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
2232025-09-03T04:15:07.389ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
2242025-09-03T04:15:07.389ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2252025-09-03T04:15:07.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
2262025-09-03T04:15:07.389ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
2272025-09-03T04:15:07.389ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 36.04521ms iteration = 23 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
2282025-09-03T04:15:07.398ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
2292025-09-03T04:15:07.398ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2302025-09-03T04:15:07.398ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 45.051827ms iteration = 23 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
2312025-09-03T04:15:07.402ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2322025-09-03T04:15:07.402ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2332025-09-03T04:15:07.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2342025-09-03T04:15:07.413ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2352025-09-03T04:15:07.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
2362025-09-03T04:15:07.428ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2372025-09-03T04:15:07.433ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
2382025-09-03T04:15:07.433ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2392025-09-03T04:15:07.433ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
2402025-09-03T04:15:07.449ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 95.687703ms iteration = 23 status = {"errors":[],"requests_created_ok":[]}
2412025-09-03T04:15:07.452ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 60e73e54-9659-486a-96ba-b69df28d8796 original_time_created = 2025-09-03 04:03:56.692551 UTC target_id = 60e73e54-9659-486a-96ba-b69df28d8796 time_created = 2025-09-03 04:03:56.692551 UTC
2422025-09-03T04:15:07.453ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 99.39294ms iteration = 67 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"60e73e54-9659-486a-96ba-b69df28d8796","time_created":"2025-09-03T04:03:56.692551Z"}
2432025-09-03T04:15:07.453ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2442025-09-03T04:15:07.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2452025-09-03T04:15:07.456ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2462025-09-03T04:15:07.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2472025-09-03T04:15:07.459ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
2482025-09-03T04:15:07.459ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = cf400120-81cc-41d9-8da6-7e88a5d7c686 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
2492025-09-03T04:15:07.459ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2502025-09-03T04:15:07.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2512025-09-03T04:15:07.460ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 106.673752ms iteration = 23 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
2522025-09-03T04:15:07.461ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
2532025-09-03T04:15:07.462ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2542025-09-03T04:15:07.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
2552025-09-03T04:15:07.464ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 110.285629ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
2562025-09-03T04:15:07.466ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
2572025-09-03T04:15:07.466ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2582025-09-03T04:15:07.466ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2592025-09-03T04:15:07.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
2602025-09-03T04:15:07.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2612025-09-03T04:15:07.468ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 115.268174ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
2622025-09-03T04:15:07.470ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = cf400120-81cc-41d9-8da6-7e88a5d7c686 dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
2632025-09-03T04:15:07.470ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
2642025-09-03T04:15:07.470ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-09-03 04:04:24.694355 UTC dns_group = internal generation = 1
2652025-09-03T04:15:07.470ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 120.336151ms iteration = 12 status = {"generation":1}
2662025-09-03T04:15:07.472ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
2672025-09-03T04:15:07.472ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 0a6d548b-6435-4b7d-951e-09ceac6a4eb0 dns_zone_name = oxide.test found_so_far = 0 version = 2
2682025-09-03T04:15:07.472ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2692025-09-03T04:15:07.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2702025-09-03T04:15:07.474ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 120.295871ms iteration = 23 status = {"errors":[],"garbage_collect_requested":[]}
2712025-09-03T04:15:07.474ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
2722025-09-03T04:15:07.474ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
2732025-09-03T04:15:07.474ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 120.564883ms iteration = 12 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
2742025-09-03T04:15:07.474ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2752025-09-03T04:15:07.477ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2762025-09-03T04:15:07.477ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
2772025-09-03T04:15:07.477ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2782025-09-03T04:15:07.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
2792025-09-03T04:15:07.478ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2802025-09-03T04:15:07.478ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2812025-09-03T04:15:07.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2822025-09-03T04:15:07.480ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2832025-09-03T04:15:07.480ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
2842025-09-03T04:15:07.480ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2852025-09-03T04:15:07.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
2862025-09-03T04:15:07.481ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2872025-09-03T04:15:07.481ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 0a6d548b-6435-4b7d-951e-09ceac6a4eb0 dns_zone_name = oxide.test found_so_far = 4 version = 2
2882025-09-03T04:15:07.481ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
2892025-09-03T04:15:07.481ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-09-03 04:04:25.477141 UTC dns_group = external generation = 2
2902025-09-03T04:15:07.481ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 130.990809ms iteration = 12 status = {"generation":2}
2912025-09-03T04:15:07.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2922025-09-03T04:15:07.484ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2932025-09-03T04:15:07.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2942025-09-03T04:15:07.486ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 132.62628ms iteration = 25 status = {"physical_disks_added":0}
2952025-09-03T04:15:07.487ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2962025-09-03T04:15:07.491ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2972025-09-03T04:15:07.492ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 6ba5229c-c111-45f6-88aa-7f00b514c681, lookup_type: ById(6ba5229c-c111-45f6-88aa-7f00b514c681) } result = Ok(())
2982025-09-03T04:15:07.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
2992025-09-03T04:15:07.492ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3002025-09-03T04:15:07.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3012025-09-03T04:15:07.494ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 140.962781ms iteration = 23 status = 0
3022025-09-03T04:15:07.498ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3032025-09-03T04:15:07.501ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
3042025-09-03T04:15:07.501ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3052025-09-03T04:15:07.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
3062025-09-03T04:15:07.548ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
3072025-09-03T04:15:07.548ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
3082025-09-03T04:15:07.548ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 194.381768ms iteration = 23 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
3092025-09-03T04:15:07.559ZINFOnexus (BackgroundTasks): VPC route manager sled 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
3102025-09-03T04:15:07.559ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
3112025-09-03T04:15:07.559ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "eb0f7ac1-d898-4ccb-b165-318f30b19f8a", "content-length": "372", "date": "Wed, 03 Sep 2025 04:15:07 GMT"} })
3122025-09-03T04:15:07.559ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3132025-09-03T04:15:07.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3142025-09-03T04:15:07.562ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3152025-09-03T04:15:07.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
3162025-09-03T04:15:07.570ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3172025-09-03T04:15:07.570ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
3182025-09-03T04:15:07.570ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 217.177023ms iteration = 12 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
3192025-09-03T04:15:07.571ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
3202025-09-03T04:15:07.572ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3212025-09-03T04:15:07.573ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-09-03T04:04:05.025624Z, time_modified: 2025-09-03T04:04:05.025624Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1, db_silo: Silo { identity: SiloIdentity { id: 1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-09-03T04:04:25.421556Z, time_modified: 2025-09-03T04:04:25.421556Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "88177654dc1555d3c82c6767dd227356fa644067c4cbd354dce9bb9fa68212ac" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1, db_silo: Silo { identity: SiloIdentity { id: 1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-09-03T04:04:25.421556Z, time_modified: 2025-09-03T04:04:25.421556Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "88177654dc1555d3c82c6767dd227356fa644067c4cbd354dce9bb9fa68212ac" }] }) }
3222025-09-03T04:15:07.573ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 223.400688ms iteration = 12 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1","tls_certs":["88177654dc1555d3c82c6767dd227356fa644067c4cbd354dce9bb9fa68212ac"]}},"default_endpoint":{"silo_id":"1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1","tls_certs":["88177654dc1555d3c82c6767dd227356fa644067c4cbd354dce9bb9fa68212ac"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
3232025-09-03T04:15:07.573ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
3242025-09-03T04:15:07.573ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
3252025-09-03T04:15:07.574ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "83f365c2-5f35-4d98-b5da-f50e6d75a52d", "date": "Wed, 03 Sep 2025 04:15:07 GMT"} })
3262025-09-03T04:15:07.574ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3272025-09-03T04:15:07.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3282025-09-03T04:15:07.574ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
3292025-09-03T04:15:07.574ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3302025-09-03T04:15:07.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
3312025-09-03T04:15:07.576ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3322025-09-03T04:15:07.577ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3332025-09-03T04:15:07.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3342025-09-03T04:15:07.579ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
3352025-09-03T04:15:07.579ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3362025-09-03T04:15:07.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
3372025-09-03T04:15:07.580ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3382025-09-03T04:15:07.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3392025-09-03T04:15:07.583ZINFOnexus (BackgroundTasks): VPC route manager sled 6ba5229c-c111-45f6-88aa-7f00b514c681 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
3402025-09-03T04:15:07.583ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
3412025-09-03T04:15:07.583ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "777ebc86-6b2c-4eb9-a357-a3b3f0571a70", "date": "Wed, 03 Sep 2025 04:15:07 GMT"} })
3422025-09-03T04:15:07.584ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 231.365795ms iteration = 23 status = {}
3432025-09-03T04:15:07.593ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1214 zone_address = fd00:1122:3344:101::2
3442025-09-03T04:15:07.593ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3452025-09-03T04:15:07.594ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1d5b44a-690a-4195-a387-a56621a1df60", "content-length": "26", "date": "Wed, 03 Sep 2025 04:15:07 GMT"} })
3462025-09-03T04:15:07.594ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1217 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3472025-09-03T04:15:07.595ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1251 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3482025-09-03T04:15:07.667ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
3492025-09-03T04:15:07.671ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1214 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 zone_address = fd00:1122:3344:101::2
3502025-09-03T04:15:07.671ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3512025-09-03T04:15:07.671ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c39c201f-e7fa-45d3-ae98-cad8b50c67a0", "content-length": "26", "date": "Wed, 03 Sep 2025 04:15:07 GMT"} })
3522025-09-03T04:15:07.671ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1217 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3532025-09-03T04:15:07.673ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1251 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40
3542025-09-03T04:15:07.686ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
3552025-09-03T04:15:07.695ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1214 zone_address = fd00:1122:3344:101::2
3562025-09-03T04:15:07.695ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3572025-09-03T04:15:07.695ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "dedceff1-8406-4844-8042-532380a603cb", "content-length": "26", "date": "Wed, 03 Sep 2025 04:15:07 GMT"} })
3582025-09-03T04:15:07.695ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1217 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3592025-09-03T04:15:07.697ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1251 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3602025-09-03T04:15:07.739ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
3612025-09-03T04:15:07.739ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "feb09eed-c5f2-49e0-8ce3-d88a0ade4b02", "content-length": "691", "date": "Wed, 03 Sep 2025 04:15:07 GMT"} })
3622025-09-03T04:15:07.739ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
3632025-09-03T04:15:07.739ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
3642025-09-03T04:15:07.739ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
3652025-09-03T04:15:07.740ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 387.306264ms iteration = 23 status = {}
3662025-09-03T04:15:07.751ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 401.289566ms iteration = 23 status = {}
3672025-09-03T04:15:07.807ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
3682025-09-03T04:15:07.896ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
3692025-09-03T04:15:07.897ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "441af528-d0bd-4952-b760-fb09bc8c3992", "date": "Wed, 03 Sep 2025 04:15:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
3702025-09-03T04:15:07.897ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
3712025-09-03T04:15:07.898ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "919b4c48-e1d1-4d5f-9801-db11d7e9b6a7", "date": "Wed, 03 Sep 2025 04:15:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
3722025-09-03T04:15:07.898ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
3732025-09-03T04:15:07.901ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
3742025-09-03T04:15:07.901ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 551.263111ms iteration = 14 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
3752025-09-03T04:15:07.903ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
3762025-09-03T04:15:07.904ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "9e22158c-6c2c-4f94-9533-2cc752b2a3e2", "date": "Wed, 03 Sep 2025 04:15:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
3772025-09-03T04:15:07.904ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
3782025-09-03T04:15:07.906ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
3792025-09-03T04:15:07.906ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "12fe1e50-e643-4420-a411-64c47034b85c", "date": "Wed, 03 Sep 2025 04:15:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
3802025-09-03T04:15:07.906ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
3812025-09-03T04:15:07.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
3822025-09-03T04:15:07.908ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "2b24596b-debe-44e4-98b5-58eff9caac72", "date": "Wed, 03 Sep 2025 04:15:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
3832025-09-03T04:15:07.908ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
3842025-09-03T04:15:07.908ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 558.842466ms iteration = 23 status = null
3852025-09-03T04:15:07.910ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
3862025-09-03T04:15:07.910ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 560.72176ms iteration = 14 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
3872025-09-03T04:15:07.931ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3882025-09-03T04:15:07.932ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3892025-09-03T04:15:07.934ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 6d67bb72-7d81-4e35-9440-7d645a356abd, rack_id: b1c08780-6eb9-4097-acea-f38d0539ff40, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(2c3aab11-bea9-412a-a4d5-9f98fa9ac796) } rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch_location = Switch0
3902025-09-03T04:15:07.934ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3912025-09-03T04:15:07.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3922025-09-03T04:15:07.947ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
3932025-09-03T04:15:07.947ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 2c3aab11-bea9-412a-a4d5-9f98fa9ac796
3942025-09-03T04:15:07.947ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
3952025-09-03T04:15:07.947ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
3962025-09-03T04:15:07.947ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3972025-09-03T04:15:07.947ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3982025-09-03T04:15:07.947ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3992025-09-03T04:15:07.947ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
4002025-09-03T04:15:07.948ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4012025-09-03T04:15:07.948ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
4022025-09-03T04:15:07.948ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40
4032025-09-03T04:15:07.948ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4042025-09-03T04:15:07.948ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4052025-09-03T04:15:07.950ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch_location = Switch0
4062025-09-03T04:15:07.950ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
4072025-09-03T04:15:08.091ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "3b47b813-2e8e-4ede-8c77-910b8d1338df", "date": "Wed, 03 Sep 2025 04:15:07 GMT"} })
4082025-09-03T04:15:08.091ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch = Switch0
4092025-09-03T04:15:08.091ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
4102025-09-03T04:15:08.097ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "cfca5bf4-7b2d-4a4e-8ef2-a487a2e0aab7", "content-length": "682", "date": "Wed, 03 Sep 2025 04:15:07 GMT"} })
4112025-09-03T04:15:08.097ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4122025-09-03T04:15:08.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4132025-09-03T04:15:08.102ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4142025-09-03T04:15:08.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4152025-09-03T04:15:08.106ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4162025-09-03T04:15:08.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4172025-09-03T04:15:08.108ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4182025-09-03T04:15:08.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4192025-09-03T04:15:08.114ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 761.182701ms iteration = 23 status = {}
4202025-09-03T04:15:08.357ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 4
4212025-09-03T04:15:08.357ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221
4222025-09-03T04:15:08.387ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
4232025-09-03T04:15:08.387ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::a]:33061
4242025-09-03T04:15:08.387ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:33061 req_id = 6e4b4921-b570-4174-b9b2-ee2f7d1708f0 roles = RoleSet { roles: {} } uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
4252025-09-03T04:15:08.388ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:33061 req_id = 6e4b4921-b570-4174-b9b2-ee2f7d1708f0 resource = Database result = Ok(()) uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
4262025-09-03T04:15:08.428ZINFOnexus: assigned collector to new producer collector_id = 5ae9222c-aa77-4bb7-9fcb-c8d01163fee8 file = nexus/src/app/oximeter.rs:76 producer_id = fa7984b8-4f16-449d-b4e0-b61c280b0e91
4272025-09-03T04:15:08.428ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 40251 local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:33061 req_id = 6e4b4921-b570-4174-b9b2-ee2f7d1708f0 response_code = 201 uri = /metrics/producers
4282025-09-03T04:15:08.428ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "6e4b4921-b570-4174-b9b2-ee2f7d1708f0", "content-length": "41", "date": "Wed, 03 Sep 2025 04:15:08 GMT"} })
4292025-09-03T04:15:08.429ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
4302025-09-03T04:15:09.652ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 325 local_addr = [fd00:1122:3344:101::a]:33204 method = GET remote_addr = [fd00:1122:3344:101::d]:55526 req_id = 2076d34e-0fc4-42ac-840b-2d963ed169c2 response_code = 200 uri = /fa7984b8-4f16-449d-b4e0-b61c280b0e91
4312025-09-03T04:15:12.352ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 134 reason = Timeout
4322025-09-03T04:15:12.352ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
4332025-09-03T04:15:12.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
4342025-09-03T04:15:12.354ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4352025-09-03T04:15:12.357ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
4362025-09-03T04:15:12.357ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
4372025-09-03T04:15:12.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
4382025-09-03T04:15:12.359ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
4392025-09-03T04:15:12.359ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.623078ms iteration = 134 status = {"chicken_switches_updated":false}
4402025-09-03T04:15:17.353ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 68 reason = Timeout
4412025-09-03T04:15:17.353ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4422025-09-03T04:15:17.353ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 68 reason = Timeout
4432025-09-03T04:15:17.353ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 135 reason = Timeout
4442025-09-03T04:15:17.353ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4452025-09-03T04:15:17.353ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
4462025-09-03T04:15:17.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4472025-09-03T04:15:17.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
4482025-09-03T04:15:17.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
4492025-09-03T04:15:17.355ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4502025-09-03T04:15:17.355ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4512025-09-03T04:15:17.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4522025-09-03T04:15:17.356ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4532025-09-03T04:15:17.357ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.09801ms iteration = 68 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
4542025-09-03T04:15:17.359ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
4552025-09-03T04:15:17.359ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
4562025-09-03T04:15:17.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
4572025-09-03T04:15:17.359ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4582025-09-03T04:15:17.359ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4592025-09-03T04:15:17.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
4602025-09-03T04:15:17.361ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
4612025-09-03T04:15:17.361ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.459564ms iteration = 135 status = {"chicken_switches_updated":false}
4622025-09-03T04:15:17.361ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4632025-09-03T04:15:17.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
4642025-09-03T04:15:17.362ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4652025-09-03T04:15:17.365ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 60e73e54-9659-486a-96ba-b69df28d8796, lookup_type: ById(60e73e54-9659-486a-96ba-b69df28d8796) } result = Ok(())
4662025-09-03T04:15:17.365ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4672025-09-03T04:15:17.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
4682025-09-03T04:15:17.378ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 60e73e54-9659-486a-96ba-b69df28d8796 original_time_created = 2025-09-03 04:03:56.692551 UTC target_id = 60e73e54-9659-486a-96ba-b69df28d8796 time_created = 2025-09-03 04:03:56.692551 UTC
4692025-09-03T04:15:17.378ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.692379ms iteration = 68 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"60e73e54-9659-486a-96ba-b69df28d8796","time_created":"2025-09-03T04:03:56.692551Z"}
4702025-09-03T04:15:19.652ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 402 local_addr = [fd00:1122:3344:101::a]:33204 method = GET remote_addr = [fd00:1122:3344:101::d]:55526 req_id = d49ddfae-c5ea-4ff8-934a-dad5a62ebfa1 response_code = 200 uri = /fa7984b8-4f16-449d-b4e0-b61c280b0e91
4712025-09-03T04:15:22.353ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 136 reason = Timeout
4722025-09-03T04:15:22.353ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
4732025-09-03T04:15:22.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
4742025-09-03T04:15:22.356ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4752025-09-03T04:15:22.358ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
4762025-09-03T04:15:22.358ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
4772025-09-03T04:15:22.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
4782025-09-03T04:15:22.361ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
4792025-09-03T04:15:22.361ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.711146ms iteration = 136 status = {"chicken_switches_updated":false}
4802025-09-03T04:15:27.353ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 69 reason = Timeout
4812025-09-03T04:15:27.353ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4822025-09-03T04:15:27.353ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 69 reason = Timeout
4832025-09-03T04:15:27.353ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4842025-09-03T04:15:27.353ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 137 reason = Timeout
4852025-09-03T04:15:27.353ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
4862025-09-03T04:15:27.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4872025-09-03T04:15:27.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
4882025-09-03T04:15:27.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
4892025-09-03T04:15:27.355ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4902025-09-03T04:15:27.355ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4912025-09-03T04:15:27.355ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4922025-09-03T04:15:27.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4932025-09-03T04:15:27.357ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.999129ms iteration = 69 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
4942025-09-03T04:15:27.359ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4952025-09-03T04:15:27.359ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4962025-09-03T04:15:27.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
4972025-09-03T04:15:27.359ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
4982025-09-03T04:15:27.359ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
4992025-09-03T04:15:27.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
5002025-09-03T04:15:27.361ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5012025-09-03T04:15:27.361ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
5022025-09-03T04:15:27.361ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.087038ms iteration = 137 status = {"chicken_switches_updated":false}
5032025-09-03T04:15:27.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
5042025-09-03T04:15:27.362ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5052025-09-03T04:15:27.365ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 60e73e54-9659-486a-96ba-b69df28d8796, lookup_type: ById(60e73e54-9659-486a-96ba-b69df28d8796) } result = Ok(())
5062025-09-03T04:15:27.365ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5072025-09-03T04:15:27.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
5082025-09-03T04:15:27.377ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 60e73e54-9659-486a-96ba-b69df28d8796 original_time_created = 2025-09-03 04:03:56.692551 UTC target_id = 60e73e54-9659-486a-96ba-b69df28d8796 time_created = 2025-09-03 04:03:56.692551 UTC
5092025-09-03T04:15:27.377ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.553898ms iteration = 69 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"60e73e54-9659-486a-96ba-b69df28d8796","time_created":"2025-09-03T04:03:56.692551Z"}
5102025-09-03T04:15:29.652ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 381 local_addr = [fd00:1122:3344:101::a]:33204 method = GET remote_addr = [fd00:1122:3344:101::d]:55526 req_id = 097b7c0f-4b6b-4374-a325-13c5ab1e559e response_code = 200 uri = /fa7984b8-4f16-449d-b4e0-b61c280b0e91
5112025-09-03T04:15:32.353ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 138 reason = Timeout
5122025-09-03T04:15:32.353ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
5132025-09-03T04:15:32.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
5142025-09-03T04:15:32.355ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5152025-09-03T04:15:32.358ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
5162025-09-03T04:15:32.358ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
5172025-09-03T04:15:32.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
5182025-09-03T04:15:32.360ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
5192025-09-03T04:15:32.360ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.246642ms iteration = 138 status = {"chicken_switches_updated":false}
5202025-09-03T04:15:34.645ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::d]:52447
5212025-09-03T04:15:34.646ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:52447 req_id = 7d5c7906-58fd-4823-bd6f-ffbf2cac6e41 roles = RoleSet { roles: {} } uri = /metrics/collectors/5ae9222c-aa77-4bb7-9fcb-c8d01163fee8/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
5222025-09-03T04:15:34.646ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:52447 req_id = 7d5c7906-58fd-4823-bd6f-ffbf2cac6e41 resource = Database result = Ok(()) uri = /metrics/collectors/5ae9222c-aa77-4bb7-9fcb-c8d01163fee8/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
5232025-09-03T04:15:34.648ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2479 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:52447 req_id = 7d5c7906-58fd-4823-bd6f-ffbf2cac6e41 response_code = 200 uri = /metrics/collectors/5ae9222c-aa77-4bb7-9fcb-c8d01163fee8/producers?sort_by=id_ascending
5242025-09-03T04:15:34.648ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:52447 req_id = 199c6b55-958c-4149-a4fd-0102e0dc9dae roles = RoleSet { roles: {} } uri = /metrics/collectors/5ae9222c-aa77-4bb7-9fcb-c8d01163fee8/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmYTk5Mjg3Mi1hNjlkLTQ3NjAtYjM3Zi0xYmQ0ZmU0MTI2ZWYifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
5252025-09-03T04:15:34.649ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:52447 req_id = 199c6b55-958c-4149-a4fd-0102e0dc9dae resource = Database result = Ok(()) uri = /metrics/collectors/5ae9222c-aa77-4bb7-9fcb-c8d01163fee8/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmYTk5Mjg3Mi1hNjlkLTQ3NjAtYjM3Zi0xYmQ0ZmU0MTI2ZWYifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
5262025-09-03T04:15:34.650ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2241 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:52447 req_id = 199c6b55-958c-4149-a4fd-0102e0dc9dae response_code = 200 uri = /metrics/collectors/5ae9222c-aa77-4bb7-9fcb-c8d01163fee8/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmYTk5Mjg3Mi1hNjlkLTQ3NjAtYjM3Zi0xYmQ0ZmU0MTI2ZWYifX0%3D
5272025-09-03T04:15:37.351ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 24 reason = Timeout
5282025-09-03T04:15:37.351ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 24 reason = Timeout
5292025-09-03T04:15:37.351ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
5302025-09-03T04:15:37.351ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 24 reason = Timeout
5312025-09-03T04:15:37.351ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
5322025-09-03T04:15:37.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
5332025-09-03T04:15:37.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
5342025-09-03T04:15:37.354ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 24 reason = Timeout
5352025-09-03T04:15:37.354ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 18.5\u{b5}s iteration = 24 status = {"disabled":true,"errors":[],"sps":[]}
5362025-09-03T04:15:37.354ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 24 reason = Timeout
5372025-09-03T04:15:37.354ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
5382025-09-03T04:15:37.354ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 26 reason = Timeout
5392025-09-03T04:15:37.354ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5402025-09-03T04:15:37.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
5412025-09-03T04:15:37.354ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 24 reason = Timeout
5422025-09-03T04:15:37.354ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5432025-09-03T04:15:37.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5442025-09-03T04:15:37.355ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 24 reason = Timeout
5452025-09-03T04:15:37.355ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5462025-09-03T04:15:37.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5472025-09-03T04:15:37.355ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 24 reason = Timeout
5482025-09-03T04:15:37.355ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
5492025-09-03T04:15:37.355ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5502025-09-03T04:15:37.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
5512025-09-03T04:15:37.355ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 24 reason = Timeout
5522025-09-03T04:15:37.355ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
5532025-09-03T04:15:37.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
5542025-09-03T04:15:37.355ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 24 reason = Timeout
5552025-09-03T04:15:37.355ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
5562025-09-03T04:15:37.355ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5572025-09-03T04:15:37.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
5582025-09-03T04:15:37.357ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 24 reason = Timeout
5592025-09-03T04:15:37.357ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
5602025-09-03T04:15:37.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
5612025-09-03T04:15:37.357ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 24 reason = Timeout
5622025-09-03T04:15:37.357ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5632025-09-03T04:15:37.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5642025-09-03T04:15:37.357ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 24 reason = Timeout
5652025-09-03T04:15:37.357ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
5662025-09-03T04:15:37.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
5672025-09-03T04:15:37.357ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 24 reason = Timeout
5682025-09-03T04:15:37.357ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
5692025-09-03T04:15:37.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5702025-09-03T04:15:37.357ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 24 reason = Timeout
5712025-09-03T04:15:37.357ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5722025-09-03T04:15:37.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
5732025-09-03T04:15:37.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
5742025-09-03T04:15:37.357ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 24 reason = Timeout
5752025-09-03T04:15:37.357ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5762025-09-03T04:15:37.357ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 24 reason = Timeout
5772025-09-03T04:15:37.357ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
5782025-09-03T04:15:37.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5792025-09-03T04:15:37.357ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 70 reason = Timeout
5802025-09-03T04:15:37.357ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5812025-09-03T04:15:37.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
5822025-09-03T04:15:37.357ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 139 reason = Timeout
5832025-09-03T04:15:37.357ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
5842025-09-03T04:15:37.357ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 70 reason = Timeout
5852025-09-03T04:15:37.357ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5862025-09-03T04:15:37.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5872025-09-03T04:15:37.358ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
5882025-09-03T04:15:37.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
5892025-09-03T04:15:37.360ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1749, tv_nsec: 32502626 } })
5902025-09-03T04:15:37.360ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1206
5912025-09-03T04:15:37.361ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 9.956743ms iteration = 24 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
5922025-09-03T04:15:37.361ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5932025-09-03T04:15:37.361ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5942025-09-03T04:15:37.361ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5952025-09-03T04:15:37.361ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5962025-09-03T04:15:37.361ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5972025-09-03T04:15:37.361ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5982025-09-03T04:15:37.362ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5992025-09-03T04:15:37.362ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6002025-09-03T04:15:37.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6012025-09-03T04:15:37.363ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 9.031515ms iteration = 24 status = {"errors":[],"finish_invoked_ok":[]}
6022025-09-03T04:15:37.363ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1749, tv_nsec: 35892840 } })
6032025-09-03T04:15:37.364ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1206
6042025-09-03T04:15:37.364ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6052025-09-03T04:15:37.364ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
6062025-09-03T04:15:37.364ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6072025-09-03T04:15:37.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
6082025-09-03T04:15:37.365ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6092025-09-03T04:15:37.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6102025-09-03T04:15:37.367ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
6112025-09-03T04:15:37.367ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
6122025-09-03T04:15:37.367ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
6132025-09-03T04:15:37.367ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
6142025-09-03T04:15:37.367ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
6152025-09-03T04:15:37.367ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6162025-09-03T04:15:37.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
6172025-09-03T04:15:37.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
6182025-09-03T04:15:37.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6192025-09-03T04:15:37.368ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6202025-09-03T04:15:37.368ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6212025-09-03T04:15:37.369ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 12.802313ms iteration = 24 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
6222025-09-03T04:15:37.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
6232025-09-03T04:15:37.369ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
6242025-09-03T04:15:37.369ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
6252025-09-03T04:15:37.369ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
6262025-09-03T04:15:37.369ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
6272025-09-03T04:15:37.369ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6282025-09-03T04:15:37.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
6292025-09-03T04:15:37.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
6302025-09-03T04:15:37.369ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6312025-09-03T04:15:37.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6322025-09-03T04:15:37.369ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
6332025-09-03T04:15:37.369ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6342025-09-03T04:15:37.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
6352025-09-03T04:15:37.370ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
6362025-09-03T04:15:37.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
6372025-09-03T04:15:37.370ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
6382025-09-03T04:15:37.370ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6392025-09-03T04:15:37.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6402025-09-03T04:15:37.371ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6412025-09-03T04:15:37.372ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 15.26427ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
6422025-09-03T04:15:37.372ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6432025-09-03T04:15:37.372ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6442025-09-03T04:15:37.373ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6452025-09-03T04:15:37.373ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
6462025-09-03T04:15:37.373ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 15.947196ms iteration = 139 status = {"chicken_switches_updated":false}
6472025-09-03T04:15:37.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
6482025-09-03T04:15:37.374ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1749, tv_nsec: 46813119 } })
6492025-09-03T04:15:37.376ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
6502025-09-03T04:15:37.376ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6512025-09-03T04:15:37.376ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6522025-09-03T04:15:37.376ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1206 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40
6532025-09-03T04:15:37.376ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6542025-09-03T04:15:37.381ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
6552025-09-03T04:15:37.381ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
6562025-09-03T04:15:37.381ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
6572025-09-03T04:15:37.381ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6582025-09-03T04:15:37.382ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6592025-09-03T04:15:37.383ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6602025-09-03T04:15:37.383ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6612025-09-03T04:15:37.384ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 28.554986ms iteration = 24 status = {"errors":[],"requests_created_ok":[]}
6622025-09-03T04:15:37.386ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
6632025-09-03T04:15:37.386ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6642025-09-03T04:15:37.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6652025-09-03T04:15:37.387ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 60e73e54-9659-486a-96ba-b69df28d8796, lookup_type: ById(60e73e54-9659-486a-96ba-b69df28d8796) } result = Ok(())
6662025-09-03T04:15:37.387ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6672025-09-03T04:15:37.387ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
6682025-09-03T04:15:37.391ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6692025-09-03T04:15:37.395ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
6702025-09-03T04:15:37.395ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6712025-09-03T04:15:37.396ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6722025-09-03T04:15:37.422ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 60e73e54-9659-486a-96ba-b69df28d8796 original_time_created = 2025-09-03 04:03:56.692551 UTC target_id = 60e73e54-9659-486a-96ba-b69df28d8796 time_created = 2025-09-03 04:03:56.692551 UTC
6732025-09-03T04:15:37.422ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 64.824969ms iteration = 70 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"60e73e54-9659-486a-96ba-b69df28d8796","time_created":"2025-09-03T04:03:56.692551Z"}
6742025-09-03T04:15:37.456ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6752025-09-03T04:15:37.456ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
6762025-09-03T04:15:37.456ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 101.180563ms iteration = 24 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
6772025-09-03T04:15:37.457ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 100.036345ms iteration = 24 status = {"errors":[],"garbage_collect_requested":[]}
6782025-09-03T04:15:37.459ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
6792025-09-03T04:15:37.459ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6802025-09-03T04:15:37.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
6812025-09-03T04:15:37.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6822025-09-03T04:15:37.463ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6832025-09-03T04:15:37.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6842025-09-03T04:15:37.465ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6852025-09-03T04:15:37.469ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6862025-09-03T04:15:37.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6872025-09-03T04:15:37.470ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
6882025-09-03T04:15:37.470ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 115.437776ms iteration = 24 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
6892025-09-03T04:15:37.471ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 6ba5229c-c111-45f6-88aa-7f00b514c681, lookup_type: ById(6ba5229c-c111-45f6-88aa-7f00b514c681) } result = Ok(())
6902025-09-03T04:15:37.471ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6912025-09-03T04:15:37.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6922025-09-03T04:15:37.474ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 119.757328ms iteration = 24 status = 0
6932025-09-03T04:15:37.475ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 117.754543ms iteration = 70 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6942025-09-03T04:15:37.507ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1214 zone_address = fd00:1122:3344:101::2
6952025-09-03T04:15:37.507ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6962025-09-03T04:15:37.508ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a24bda25-fd8b-4e8d-bcec-c1b38a21dda1", "content-length": "26", "date": "Wed, 03 Sep 2025 04:15:37 GMT"} })
6972025-09-03T04:15:37.508ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1217 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6982025-09-03T04:15:37.508ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1214 zone_address = fd00:1122:3344:101::2
6992025-09-03T04:15:37.508ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7002025-09-03T04:15:37.508ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1214 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 zone_address = fd00:1122:3344:101::2
7012025-09-03T04:15:37.508ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7022025-09-03T04:15:37.508ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "86e7740b-5576-4ac1-bcc7-deaa3f44dca7", "content-length": "26", "date": "Wed, 03 Sep 2025 04:15:37 GMT"} })
7032025-09-03T04:15:37.508ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "49b07be3-c1e6-4565-b635-fa38bfb11187", "content-length": "26", "date": "Wed, 03 Sep 2025 04:15:37 GMT"} })
7042025-09-03T04:15:37.508ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1217 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7052025-09-03T04:15:37.508ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1217 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7062025-09-03T04:15:37.512ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1251 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
7072025-09-03T04:15:37.512ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1251 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40
7082025-09-03T04:15:37.512ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1251 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
7092025-09-03T04:15:37.529ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 173.150964ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
7102025-09-03T04:15:37.547ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = instance_watcher body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vmms/8f17b50d-9850-4af8-b7e4-8f461e962229/state
7112025-09-03T04:15:37.587ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7122025-09-03T04:15:37.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
7132025-09-03T04:15:37.599ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7142025-09-03T04:15:37.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
7152025-09-03T04:15:37.602ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7162025-09-03T04:15:37.604ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7172025-09-03T04:15:37.604ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7182025-09-03T04:15:37.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
7192025-09-03T04:15:37.606ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7202025-09-03T04:15:37.608ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7212025-09-03T04:15:37.608ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7222025-09-03T04:15:37.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
7232025-09-03T04:15:37.609ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 255.729142ms iteration = 26 status = {"physical_disks_added":0}
7242025-09-03T04:15:37.631ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 279.969047ms iteration = 24 status = {}
7252025-09-03T04:15:37.635ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
7262025-09-03T04:15:37.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
7272025-09-03T04:15:37.637ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 286.289883ms iteration = 24 status = null
7282025-09-03T04:15:37.652ZINFOnexus (BackgroundTasks): VPC route manager sled 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
7292025-09-03T04:15:37.652ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
7302025-09-03T04:15:37.652ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "79eff2d6-1647-4411-8cfa-ac936040ce38", "content-length": "525", "date": "Wed, 03 Sep 2025 04:15:37 GMT"} })
7312025-09-03T04:15:37.652ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7322025-09-03T04:15:37.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7332025-09-03T04:15:37.692ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
7342025-09-03T04:15:37.693ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc9f5de7-033d-46e1-bb0a-0dfafc9eb2da", "content-length": "811", "date": "Wed, 03 Sep 2025 04:15:37 GMT"} })
7352025-09-03T04:15:37.693ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {VirtualNetworkInterfaceHost { physical_host_ip: fd00:1122:3344:101::1, virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), vni: Vni(12030372) }} sled = buskin
7362025-09-03T04:15:37.693ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
7372025-09-03T04:15:37.693ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
7382025-09-03T04:15:37.695ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 338.22434ms iteration = 24 status = {}
7392025-09-03T04:15:37.745ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7402025-09-03T04:15:37.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7412025-09-03T04:15:37.749ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 6d67bb72-7d81-4e35-9440-7d645a356abd, rack_id: b1c08780-6eb9-4097-acea-f38d0539ff40, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(2c3aab11-bea9-412a-a4d5-9f98fa9ac796) } rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch_location = Switch0
7422025-09-03T04:15:37.749ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7432025-09-03T04:15:37.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7442025-09-03T04:15:37.764ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
7452025-09-03T04:15:37.764ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 2c3aab11-bea9-412a-a4d5-9f98fa9ac796
7462025-09-03T04:15:37.764ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
7472025-09-03T04:15:37.764ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
7482025-09-03T04:15:37.764ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
7492025-09-03T04:15:37.764ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
7502025-09-03T04:15:37.764ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
7512025-09-03T04:15:37.764ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
7522025-09-03T04:15:37.764ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
7532025-09-03T04:15:37.764ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
7542025-09-03T04:15:37.765ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40
7552025-09-03T04:15:37.765ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7562025-09-03T04:15:37.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7572025-09-03T04:15:37.768ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch_location = Switch0
7582025-09-03T04:15:37.768ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
7592025-09-03T04:15:37.822ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {37311ff8-5cde-4d05-aa40-7c01012a2b36: {10.151.1.176: {fadf70e0-83c4-4019-9200-6730d768ea9d}, 10.151.1.175: {fadf70e0-83c4-4019-9200-6730d768ea9d}}} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
7602025-09-03T04:15:37.822ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
7612025-09-03T04:15:37.823ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "e7b4f1b2-ffb3-4f67-9728-6b9e5aa0be31", "date": "Wed, 03 Sep 2025 04:15:37 GMT"} })
7622025-09-03T04:15:37.823ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7632025-09-03T04:15:37.823ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7642025-09-03T04:15:37.827ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7652025-09-03T04:15:37.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7662025-09-03T04:15:37.832ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7672025-09-03T04:15:37.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7682025-09-03T04:15:37.837ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7692025-09-03T04:15:37.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7702025-09-03T04:15:37.841ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7712025-09-03T04:15:37.841ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7722025-09-03T04:15:37.846ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7732025-09-03T04:15:37.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7742025-09-03T04:15:37.851ZINFOnexus (BackgroundTasks): VPC route manager sled 6ba5229c-c111-45f6-88aa-7f00b514c681 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
7752025-09-03T04:15:37.851ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7762025-09-03T04:15:37.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7772025-09-03T04:15:37.857ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7782025-09-03T04:15:37.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7792025-09-03T04:15:37.862ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7802025-09-03T04:15:37.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7812025-09-03T04:15:37.865ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7822025-09-03T04:15:37.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7832025-09-03T04:15:37.868ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7842025-09-03T04:15:37.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7852025-09-03T04:15:37.870ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7862025-09-03T04:15:37.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7872025-09-03T04:15:37.872ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7882025-09-03T04:15:37.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7892025-09-03T04:15:37.875ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7902025-09-03T04:15:37.880ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1, lookup_type: ById(1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1) }, key: d9676708-e397-40de-a3cb-d7a767a4f75b, lookup_type: ById(d9676708-e397-40de-a3cb-d7a767a4f75b) }, key: da219404-a79b-4f5a-a91a-c7c949125ac4, lookup_type: ById(da219404-a79b-4f5a-a91a-c7c949125ac4) }, key: 317fce13-5ad5-4443-8e18-870782a5042a, lookup_type: ById(317fce13-5ad5-4443-8e18-870782a5042a) } result = Ok(())
7912025-09-03T04:15:37.880ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7922025-09-03T04:15:37.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7932025-09-03T04:15:37.889ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7942025-09-03T04:15:37.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7952025-09-03T04:15:37.891ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7962025-09-03T04:15:37.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7972025-09-03T04:15:37.893ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7982025-09-03T04:15:37.898ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1, lookup_type: ById(1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1) }, key: d9676708-e397-40de-a3cb-d7a767a4f75b, lookup_type: ById(d9676708-e397-40de-a3cb-d7a767a4f75b) }, key: da219404-a79b-4f5a-a91a-c7c949125ac4, lookup_type: ById(da219404-a79b-4f5a-a91a-c7c949125ac4) }, key: 317fce13-5ad5-4443-8e18-870782a5042a, lookup_type: ById(317fce13-5ad5-4443-8e18-870782a5042a) } result = Ok(())
7992025-09-03T04:15:37.898ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8002025-09-03T04:15:37.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8012025-09-03T04:15:37.903ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8022025-09-03T04:15:37.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8032025-09-03T04:15:37.906ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8042025-09-03T04:15:37.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8052025-09-03T04:15:37.908ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8062025-09-03T04:15:37.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8072025-09-03T04:15:37.912ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8082025-09-03T04:15:37.912ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8092025-09-03T04:15:37.915ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8102025-09-03T04:15:37.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8112025-09-03T04:15:37.917ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8122025-09-03T04:15:37.918ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8132025-09-03T04:15:37.920ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8142025-09-03T04:15:37.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8152025-09-03T04:15:37.922ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8162025-09-03T04:15:37.927ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1, lookup_type: ById(1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1) }, key: d9676708-e397-40de-a3cb-d7a767a4f75b, lookup_type: ById(d9676708-e397-40de-a3cb-d7a767a4f75b) }, key: da219404-a79b-4f5a-a91a-c7c949125ac4, lookup_type: ById(da219404-a79b-4f5a-a91a-c7c949125ac4) }, key: fa799857-1c69-48bd-8804-4526a6b01654, lookup_type: ById(fa799857-1c69-48bd-8804-4526a6b01654) } result = Ok(())
8172025-09-03T04:15:37.930ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8182025-09-03T04:15:37.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8192025-09-03T04:15:37.933ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8202025-09-03T04:15:37.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8212025-09-03T04:15:37.936ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8222025-09-03T04:15:37.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8232025-09-03T04:15:37.938ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8242025-09-03T04:15:37.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8252025-09-03T04:15:37.941ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8262025-09-03T04:15:37.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8272025-09-03T04:15:37.943ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8282025-09-03T04:15:37.943ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8292025-09-03T04:15:37.946ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8302025-09-03T04:15:37.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8312025-09-03T04:15:37.947ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8322025-09-03T04:15:37.953ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1, lookup_type: ById(1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1) }, key: d9676708-e397-40de-a3cb-d7a767a4f75b, lookup_type: ById(d9676708-e397-40de-a3cb-d7a767a4f75b) }, key: da219404-a79b-4f5a-a91a-c7c949125ac4, lookup_type: ById(da219404-a79b-4f5a-a91a-c7c949125ac4) }, key: fa799857-1c69-48bd-8804-4526a6b01654, lookup_type: ByName("default") } result = Ok(())
8332025-09-03T04:15:37.953ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8342025-09-03T04:15:37.953ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8352025-09-03T04:15:37.955ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8362025-09-03T04:15:37.955ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8372025-09-03T04:15:37.958ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8382025-09-03T04:15:37.958ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8392025-09-03T04:15:37.960ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8402025-09-03T04:15:37.960ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8412025-09-03T04:15:37.966ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8422025-09-03T04:15:37.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8432025-09-03T04:15:37.968ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8442025-09-03T04:15:37.968ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8452025-09-03T04:15:37.970ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8462025-09-03T04:15:37.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8472025-09-03T04:15:37.972ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8482025-09-03T04:15:37.977ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = InternetGateway { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1, lookup_type: ById(1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1) }, key: d9676708-e397-40de-a3cb-d7a767a4f75b, lookup_type: ById(d9676708-e397-40de-a3cb-d7a767a4f75b) }, key: da219404-a79b-4f5a-a91a-c7c949125ac4, lookup_type: ById(da219404-a79b-4f5a-a91a-c7c949125ac4) }, key: fadf70e0-83c4-4019-9200-6730d768ea9d, lookup_type: ByName("default") } result = Ok(())
8492025-09-03T04:15:37.989ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
8502025-09-03T04:15:37.993ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "8cc91e07-7a32-4325-97c9-3d0c32745ba9", "date": "Wed, 03 Sep 2025 04:15:37 GMT"} })
8512025-09-03T04:15:37.995ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 640.450188ms iteration = 24 status = {}
8522025-09-03T04:15:38.150ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "57c6a7a5-d02e-4aae-8441-890cd24bb70c", "date": "Wed, 03 Sep 2025 04:15:37 GMT"} })
8532025-09-03T04:15:38.151ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch = Switch0
8542025-09-03T04:15:38.151ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
8552025-09-03T04:15:38.151ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "e8f385b0-b473-4dfb-8a73-7a5dbefa7c27", "content-length": "682", "date": "Wed, 03 Sep 2025 04:15:37 GMT"} })
8562025-09-03T04:15:38.151ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8572025-09-03T04:15:38.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8582025-09-03T04:15:38.156ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8592025-09-03T04:15:38.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8602025-09-03T04:15:38.160ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8612025-09-03T04:15:38.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8622025-09-03T04:15:38.163ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8632025-09-03T04:15:38.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8642025-09-03T04:15:38.174ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 816.825275ms iteration = 24 status = {}
8652025-09-03T04:15:39.654ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 454 local_addr = [fd00:1122:3344:101::a]:33204 method = GET remote_addr = [fd00:1122:3344:101::d]:55526 req_id = e4a541c1-63f4-48ce-8381-636c8db2f431 response_code = 200 uri = /fa7984b8-4f16-449d-b4e0-b61c280b0e91
8662025-09-03T04:15:42.354ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 140 reason = Timeout
8672025-09-03T04:15:42.354ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
8682025-09-03T04:15:42.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
8692025-09-03T04:15:42.356ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8702025-09-03T04:15:42.359ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
8712025-09-03T04:15:42.359ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
8722025-09-03T04:15:42.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
8732025-09-03T04:15:42.360ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
8742025-09-03T04:15:42.360ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.375686ms iteration = 140 status = {"chicken_switches_updated":false}
8752025-09-03T04:15:42.876ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = instance_watcher result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vmms/8f17b50d-9850-4af8-b7e4-8f461e962229/state", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac80fee3-31d7-40bb-8e9d-bc8c36f60dbe", "content-length": "131", "date": "Wed, 03 Sep 2025 04:15:42 GMT"} })
8762025-09-03T04:15:42.879ZDEBGnexus (BackgroundTasks): updating instance state background_task = instance_watcher instance_id = 93025ecf-bf8a-46be-81c6-2fe0202411bb sled_id = 6ba5229c-c111-45f6-88aa-7f00b514c681 state = SledVmmState { vmm_state: VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2025-09-03T04:15:35.131592692Z }, migration_in: None, migration_out: None } vmm_id = 8f17b50d-9850-4af8-b7e4-8f461e962229
8772025-09-03T04:15:42.879ZINFOnexus (BackgroundTasks): received new VMM runtime state from sled agent background_task = instance_watcher file = nexus/src/app/instance.rs:2140 instance_id = 93025ecf-bf8a-46be-81c6-2fe0202411bb migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 8f17b50d-9850-4af8-b7e4-8f461e962229 sled_id = 6ba5229c-c111-45f6-88aa-7f00b514c681 vmm_id = 8f17b50d-9850-4af8-b7e4-8f461e962229 vmm_state = VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2025-09-03T04:15:35.131592692Z }
8782025-09-03T04:15:42.879ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = 93025ecf-bf8a-46be-81c6-2fe0202411bb roles = RoleSet { roles: {} } sled_id = 6ba5229c-c111-45f6-88aa-7f00b514c681 vmm_id = 8f17b50d-9850-4af8-b7e4-8f461e962229
8792025-09-03T04:15:42.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher instance_id = 93025ecf-bf8a-46be-81c6-2fe0202411bb resource = Database result = Ok(()) sled_id = 6ba5229c-c111-45f6-88aa-7f00b514c681 vmm_id = 8f17b50d-9850-4af8-b7e4-8f461e962229
8802025-09-03T04:15:42.898ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 1 total_failed = 0 total_incomplete = 0 total_instances = 1 update_sagas_queued = 0
8812025-09-03T04:15:42.900ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 5.545763546s iteration = 24 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{"starting":1},"pruned_instances":0,"total_instances":1,"update_sagas_queued":0}
8822025-09-03T04:15:47.354ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 71 reason = Timeout
8832025-09-03T04:15:47.354ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 71 reason = Timeout
8842025-09-03T04:15:47.354ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 141 reason = Timeout
8852025-09-03T04:15:47.354ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8862025-09-03T04:15:47.354ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8872025-09-03T04:15:47.354ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
8882025-09-03T04:15:47.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
8892025-09-03T04:15:47.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
8902025-09-03T04:15:47.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8912025-09-03T04:15:47.356ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8922025-09-03T04:15:47.356ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8932025-09-03T04:15:47.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8942025-09-03T04:15:47.357ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8952025-09-03T04:15:47.358ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.585956ms iteration = 71 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
8962025-09-03T04:15:47.360ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
8972025-09-03T04:15:47.360ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
8982025-09-03T04:15:47.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
8992025-09-03T04:15:47.360ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9002025-09-03T04:15:47.360ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9012025-09-03T04:15:47.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
9022025-09-03T04:15:47.362ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
9032025-09-03T04:15:47.362ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.871767ms iteration = 141 status = {"chicken_switches_updated":false}
9042025-09-03T04:15:47.362ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9052025-09-03T04:15:47.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
9062025-09-03T04:15:47.364ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9072025-09-03T04:15:47.366ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 60e73e54-9659-486a-96ba-b69df28d8796, lookup_type: ById(60e73e54-9659-486a-96ba-b69df28d8796) } result = Ok(())
9082025-09-03T04:15:47.366ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9092025-09-03T04:15:47.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
9102025-09-03T04:15:47.379ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 60e73e54-9659-486a-96ba-b69df28d8796 original_time_created = 2025-09-03 04:03:56.692551 UTC target_id = 60e73e54-9659-486a-96ba-b69df28d8796 time_created = 2025-09-03 04:03:56.692551 UTC
9112025-09-03T04:15:47.379ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.89178ms iteration = 71 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"60e73e54-9659-486a-96ba-b69df28d8796","time_created":"2025-09-03T04:03:56.692551Z"}
9122025-09-03T04:15:49.656ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2814 local_addr = [fd00:1122:3344:101::a]:33204 method = GET remote_addr = [fd00:1122:3344:101::d]:55526 req_id = f5c07ea3-34bc-4cc9-8cf1-ea37006caae7 response_code = 200 uri = /fa7984b8-4f16-449d-b4e0-b61c280b0e91
9132025-09-03T04:15:52.354ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 142 reason = Timeout
9142025-09-03T04:15:52.354ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
9152025-09-03T04:15:52.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
9162025-09-03T04:15:52.356ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9172025-09-03T04:15:52.359ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
9182025-09-03T04:15:52.359ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
9192025-09-03T04:15:52.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
9202025-09-03T04:15:52.361ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
9212025-09-03T04:15:52.361ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.391964ms iteration = 142 status = {"chicken_switches_updated":false}
9222025-09-03T04:15:57.354ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 72 reason = Timeout
9232025-09-03T04:15:57.354ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9242025-09-03T04:15:57.355ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 72 reason = Timeout
9252025-09-03T04:15:57.355ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 143 reason = Timeout
9262025-09-03T04:15:57.355ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
9272025-09-03T04:15:57.355ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9282025-09-03T04:15:57.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
9292025-09-03T04:15:57.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9302025-09-03T04:15:57.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
9312025-09-03T04:15:57.356ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9322025-09-03T04:15:57.357ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9332025-09-03T04:15:57.357ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9342025-09-03T04:15:57.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9352025-09-03T04:15:57.358ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.572996ms iteration = 72 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
9362025-09-03T04:15:57.360ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
9372025-09-03T04:15:57.360ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
9382025-09-03T04:15:57.360ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9392025-09-03T04:15:57.360ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9402025-09-03T04:15:57.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
9412025-09-03T04:15:57.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
9422025-09-03T04:15:57.362ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9432025-09-03T04:15:57.363ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
9442025-09-03T04:15:57.363ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
9452025-09-03T04:15:57.363ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.22313ms iteration = 143 status = {"chicken_switches_updated":false}
9462025-09-03T04:15:57.364ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9472025-09-03T04:15:57.367ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 60e73e54-9659-486a-96ba-b69df28d8796, lookup_type: ById(60e73e54-9659-486a-96ba-b69df28d8796) } result = Ok(())
9482025-09-03T04:15:57.367ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9492025-09-03T04:15:57.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
9502025-09-03T04:15:57.380ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 60e73e54-9659-486a-96ba-b69df28d8796 original_time_created = 2025-09-03 04:03:56.692551 UTC target_id = 60e73e54-9659-486a-96ba-b69df28d8796 time_created = 2025-09-03 04:03:56.692551 UTC
9512025-09-03T04:15:57.380ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.201632ms iteration = 72 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"60e73e54-9659-486a-96ba-b69df28d8796","time_created":"2025-09-03T04:03:56.692551Z"}
9522025-09-03T04:15:59.654ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 636 local_addr = [fd00:1122:3344:101::a]:33204 method = GET remote_addr = [fd00:1122:3344:101::d]:55526 req_id = 73b7b47e-4500-4e88-9409-0f050c1e9b0a response_code = 200 uri = /fa7984b8-4f16-449d-b4e0-b61c280b0e91
9532025-09-03T04:16:02.354ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 144 reason = Timeout
9542025-09-03T04:16:02.354ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
9552025-09-03T04:16:02.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
9562025-09-03T04:16:02.357ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9572025-09-03T04:16:02.360ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
9582025-09-03T04:16:02.360ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
9592025-09-03T04:16:02.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
9602025-09-03T04:16:02.363ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
9612025-09-03T04:16:02.363ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.546472ms iteration = 144 status = {"chicken_switches_updated":false}
9622025-09-03T04:16:07.352ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 13 reason = Timeout
9632025-09-03T04:16:07.352ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 14 reason = Timeout
9642025-09-03T04:16:07.352ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
9652025-09-03T04:16:07.352ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:133 reason = blueprint disabled target_id = 60e73e54-9659-486a-96ba-b69df28d8796
9662025-09-03T04:16:07.352ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9672025-09-03T04:16:07.352ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 52.03\u{b5}s iteration = 14 status = {"enabled":false,"target_id":"60e73e54-9659-486a-96ba-b69df28d8796"}
9682025-09-03T04:16:07.352ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 13 reason = Timeout
9692025-09-03T04:16:07.352ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 25 reason = Timeout
9702025-09-03T04:16:07.352ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
9712025-09-03T04:16:07.352ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-09-03 04:06:07.352438796 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
9722025-09-03T04:16:07.352ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 25 reason = Timeout
9732025-09-03T04:16:07.352ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
9742025-09-03T04:16:07.352ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 25 reason = Timeout
9752025-09-03T04:16:07.352ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
9762025-09-03T04:16:07.352ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 13 reason = Timeout
9772025-09-03T04:16:07.352ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 13 reason = Timeout
9782025-09-03T04:16:07.352ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 15 reason = Timeout
9792025-09-03T04:16:07.352ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 13 reason = Timeout
9802025-09-03T04:16:07.352ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 13 reason = Timeout
9812025-09-03T04:16:07.352ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
9822025-09-03T04:16:07.352ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
9832025-09-03T04:16:07.352ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
9842025-09-03T04:16:07.352ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
9852025-09-03T04:16:07.352ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 15 reason = Timeout
9862025-09-03T04:16:07.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
9872025-09-03T04:16:07.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
9882025-09-03T04:16:07.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
9892025-09-03T04:16:07.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
9902025-09-03T04:16:07.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
9912025-09-03T04:16:07.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
9922025-09-03T04:16:07.353ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0d788cd6-59e1-4eee-8401-73a64a0795db.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("33692814-cb11-4468-bf51-4d6d8418d856.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("808b9b46-4a4d-447d-99d9-09db72c60cdc.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("808b9b46-4a4d-447d-99d9-09db72c60cdc.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 1779, tv_nsec: 24984734 } })
9932025-09-03T04:16:07.353ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2085efc3-987d-4d25-bf4e-dbfa977a0d96.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a853d036-9d99-4741-85fc-3823419b4ac1.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("a853d036-9d99-4741-85fc-3823419b4ac1.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1779, tv_nsec: 25198585 } })
9942025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
9952025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.47526ms iteration = 13 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353"]}
9962025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
9972025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.622892ms iteration = 13 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
9982025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
9992025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10002025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10012025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 6 reason = Timeout
10022025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 25 reason = Timeout
10032025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
10042025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
10052025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 25 reason = Timeout
10062025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
10072025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10082025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 25 reason = Timeout
10092025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
10102025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
10112025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 25 reason = Timeout
10122025-09-03T04:16:07.354ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
10132025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 13 reason = Timeout
10142025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10152025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
10162025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 25 reason = Timeout
10172025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 25 reason = Timeout
10182025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
10192025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
10202025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 13 reason = Timeout
10212025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
10222025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 25 reason = Timeout
10232025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 25 reason = Timeout
10242025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 6.38\u{b5}s iteration = 25 status = {"disabled":true,"errors":[],"sps":[]}
10252025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10262025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 145 reason = Timeout
10272025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
10282025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 73 reason = Timeout
10292025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10302025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 73 reason = Timeout
10312025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
10322025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 25 reason = Timeout
10332025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10342025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 25 reason = Timeout
10352025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
10362025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 25 reason = Timeout
10372025-09-03T04:16:07.354ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
10382025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 27 reason = Timeout
10392025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10402025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 25 reason = Timeout
10412025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
10422025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 25 reason = Timeout
10432025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
10442025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 25 reason = Timeout
10452025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
10462025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 13 reason = Timeout
10472025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
10482025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 16 reason = Timeout
10492025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
10502025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 9.2\u{b5}s iteration = 16 status = "Disabled"
10512025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 13 reason = Timeout
10522025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 13 reason = Timeout
10532025-09-03T04:16:07.355ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
10542025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
10552025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
10562025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
10572025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 13 reason = Timeout
10582025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
10592025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
10602025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
10612025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10622025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
10632025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
10642025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
10652025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
10662025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
10672025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
10682025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
10692025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
10702025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10712025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
10722025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
10732025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10742025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
10752025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10762025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
10772025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
10782025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
10792025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
10802025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
10812025-09-03T04:16:07.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
10822025-09-03T04:16:07.356ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
10832025-09-03T04:16:07.356ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1779, tv_nsec: 28183997 } })
10842025-09-03T04:16:07.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
10852025-09-03T04:16:07.356ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
10862025-09-03T04:16:07.356ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1206
10872025-09-03T04:16:07.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
10882025-09-03T04:16:07.357ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
10892025-09-03T04:16:07.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
10902025-09-03T04:16:07.357ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
10912025-09-03T04:16:07.359ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10922025-09-03T04:16:07.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
10932025-09-03T04:16:07.359ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1779, tv_nsec: 29341445 } })
10942025-09-03T04:16:07.359ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10952025-09-03T04:16:07.359ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1206
10962025-09-03T04:16:07.362ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
10972025-09-03T04:16:07.362ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
10982025-09-03T04:16:07.364ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 9.623609ms iteration = 13 status = {"errors":[],"found_port_ok":[]}
10992025-09-03T04:16:07.364ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11002025-09-03T04:16:07.364ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
11012025-09-03T04:16:07.364ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
11022025-09-03T04:16:07.364ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
11032025-09-03T04:16:07.364ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
11042025-09-03T04:16:07.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
11052025-09-03T04:16:07.368ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
11062025-09-03T04:16:07.368ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
11072025-09-03T04:16:07.368ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11082025-09-03T04:16:07.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
11092025-09-03T04:16:07.372ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-09-03 04:06:07.352438796 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
11102025-09-03T04:16:07.372ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 19.644093ms iteration = 13 status = {"expiration":"2025-09-03T04:06:07.352438796Z","pruned":[]}
11112025-09-03T04:16:07.372ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
11122025-09-03T04:16:07.372ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
11132025-09-03T04:16:07.372ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
11142025-09-03T04:16:07.375ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11152025-09-03T04:16:07.376ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
11162025-09-03T04:16:07.376ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 21.439016ms iteration = 13 status = {"by_rx":{},"error":null}
11172025-09-03T04:16:07.376ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11182025-09-03T04:16:07.376ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11192025-09-03T04:16:07.376ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11202025-09-03T04:16:07.376ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
11212025-09-03T04:16:07.376ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
11222025-09-03T04:16:07.376ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 21.928079ms iteration = 13 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
11232025-09-03T04:16:07.376ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 24.236465ms iteration = 25 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
11242025-09-03T04:16:07.378ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11252025-09-03T04:16:07.378ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
11262025-09-03T04:16:07.378ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
11272025-09-03T04:16:07.378ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
11282025-09-03T04:16:07.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
11292025-09-03T04:16:07.379ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
11302025-09-03T04:16:07.379ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11312025-09-03T04:16:07.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11322025-09-03T04:16:07.380ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
11332025-09-03T04:16:07.380ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
11342025-09-03T04:16:07.380ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
11352025-09-03T04:16:07.380ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
11362025-09-03T04:16:07.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
11372025-09-03T04:16:07.381ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
11382025-09-03T04:16:07.381ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11392025-09-03T04:16:07.381ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
11402025-09-03T04:16:07.381ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
11412025-09-03T04:16:07.381ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11422025-09-03T04:16:07.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
11432025-09-03T04:16:07.382ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
11442025-09-03T04:16:07.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
11452025-09-03T04:16:07.382ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
11462025-09-03T04:16:07.382ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
11472025-09-03T04:16:07.383ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
11482025-09-03T04:16:07.384ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
11492025-09-03T04:16:07.384ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11502025-09-03T04:16:07.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11512025-09-03T04:16:07.385ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
11522025-09-03T04:16:07.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
11532025-09-03T04:16:07.386ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11542025-09-03T04:16:07.386ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11552025-09-03T04:16:07.389ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
11562025-09-03T04:16:07.389ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
11572025-09-03T04:16:07.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
11582025-09-03T04:16:07.389ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11592025-09-03T04:16:07.391ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
11602025-09-03T04:16:07.391ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
11612025-09-03T04:16:07.391ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
11622025-09-03T04:16:07.392ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
11632025-09-03T04:16:07.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
11642025-09-03T04:16:07.392ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
11652025-09-03T04:16:07.392ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
11662025-09-03T04:16:07.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
11672025-09-03T04:16:07.394ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
11682025-09-03T04:16:07.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
11692025-09-03T04:16:07.395ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11702025-09-03T04:16:07.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11712025-09-03T04:16:07.396ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11722025-09-03T04:16:07.397ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11732025-09-03T04:16:07.397ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11742025-09-03T04:16:07.397ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
11752025-09-03T04:16:07.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
11762025-09-03T04:16:07.398ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
11772025-09-03T04:16:07.398ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
11782025-09-03T04:16:07.398ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
11792025-09-03T04:16:07.398ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11802025-09-03T04:16:07.398ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
11812025-09-03T04:16:07.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
11822025-09-03T04:16:07.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
11832025-09-03T04:16:07.400ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
11842025-09-03T04:16:07.400ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 0a6d548b-6435-4b7d-951e-09ceac6a4eb0 dns_zone_name = oxide.test found_so_far = 0 version = 2
11852025-09-03T04:16:07.400ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
11862025-09-03T04:16:07.400ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
11872025-09-03T04:16:07.400ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
11882025-09-03T04:16:07.400ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 45.890582ms iteration = 145 status = {"chicken_switches_updated":false}
11892025-09-03T04:16:07.401ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11902025-09-03T04:16:07.401ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
11912025-09-03T04:16:07.401ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
11922025-09-03T04:16:07.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
11932025-09-03T04:16:07.401ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
11942025-09-03T04:16:07.401ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11952025-09-03T04:16:07.401ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11962025-09-03T04:16:07.401ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11972025-09-03T04:16:07.401ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 46.93397ms iteration = 13 status = {"dispatched":[],"errors":[],"glob_version":"186.0.0","globs_reprocessed":{},"no_receivers":[]}
11982025-09-03T04:16:07.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11992025-09-03T04:16:07.402ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12002025-09-03T04:16:07.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12012025-09-03T04:16:07.403ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 48.752344ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
12022025-09-03T04:16:07.404ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 49.64335ms iteration = 25 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
12032025-09-03T04:16:07.404ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
12042025-09-03T04:16:07.404ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
12052025-09-03T04:16:07.404ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12062025-09-03T04:16:07.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
12072025-09-03T04:16:07.405ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
12082025-09-03T04:16:07.405ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
12092025-09-03T04:16:07.405ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 51.329961ms iteration = 25 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
12102025-09-03T04:16:07.405ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
12112025-09-03T04:16:07.405ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
12122025-09-03T04:16:07.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12132025-09-03T04:16:07.406ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
12142025-09-03T04:16:07.406ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
12152025-09-03T04:16:07.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
12162025-09-03T04:16:07.406ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
12172025-09-03T04:16:07.406ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
12182025-09-03T04:16:07.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
12192025-09-03T04:16:07.407ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12202025-09-03T04:16:07.407ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
12212025-09-03T04:16:07.407ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
12222025-09-03T04:16:07.407ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
12232025-09-03T04:16:07.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
12242025-09-03T04:16:07.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
12252025-09-03T04:16:07.410ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
12262025-09-03T04:16:07.410ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
12272025-09-03T04:16:07.410ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
12282025-09-03T04:16:07.411ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 1 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
12292025-09-03T04:16:07.411ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 56.427959ms iteration = 25 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":1,"total_instances":0,"update_sagas_queued":0}
12302025-09-03T04:16:07.411ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
12312025-09-03T04:16:07.411ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12322025-09-03T04:16:07.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12332025-09-03T04:16:07.420ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12342025-09-03T04:16:07.420ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 66.221749ms iteration = 6 status = {"nsuccess":5}
12352025-09-03T04:16:07.422ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 0a6d548b-6435-4b7d-951e-09ceac6a4eb0 dns_zone_name = oxide.test found_so_far = 4 version = 2
12362025-09-03T04:16:07.422ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
12372025-09-03T04:16:07.422ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-09-03 04:04:25.477141 UTC dns_group = external generation = 2
12382025-09-03T04:16:07.422ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 70.077678ms iteration = 13 status = {"generation":2}
12392025-09-03T04:16:07.423ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 68.494736ms iteration = 27 status = {"physical_disks_added":0}
12402025-09-03T04:16:07.424ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
12412025-09-03T04:16:07.424ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
12422025-09-03T04:16:07.424ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
12432025-09-03T04:16:07.445ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 90.783297ms iteration = 25 status = {"errors":[],"finish_invoked_ok":[]}
12442025-09-03T04:16:07.452ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 97.236814ms iteration = 25 status = {"errors":[],"requests_created_ok":[]}
12452025-09-03T04:16:07.452ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12462025-09-03T04:16:07.453ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
12472025-09-03T04:16:07.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12482025-09-03T04:16:07.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
12492025-09-03T04:16:07.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
12502025-09-03T04:16:07.457ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
12512025-09-03T04:16:07.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
12522025-09-03T04:16:07.459ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
12532025-09-03T04:16:07.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12542025-09-03T04:16:07.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
12552025-09-03T04:16:07.468ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
12562025-09-03T04:16:07.469ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
12572025-09-03T04:16:07.475ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12582025-09-03T04:16:07.476ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12592025-09-03T04:16:07.476ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
12602025-09-03T04:16:07.476ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 121.787762ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
12612025-09-03T04:16:07.479ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
12622025-09-03T04:16:07.479ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12632025-09-03T04:16:07.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12642025-09-03T04:16:07.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12652025-09-03T04:16:07.482ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 127.980117ms iteration = 25 status = {"errors":[],"garbage_collect_requested":[]}
12662025-09-03T04:16:07.483ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1779, tv_nsec: 154840524 } })
12672025-09-03T04:16:07.483ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1206 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40
12682025-09-03T04:16:07.483ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 60e73e54-9659-486a-96ba-b69df28d8796, lookup_type: ById(60e73e54-9659-486a-96ba-b69df28d8796) } result = Ok(())
12692025-09-03T04:16:07.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12702025-09-03T04:16:07.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
12712025-09-03T04:16:07.490ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
12722025-09-03T04:16:07.490ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = cf400120-81cc-41d9-8da6-7e88a5d7c686 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
12732025-09-03T04:16:07.490ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
12742025-09-03T04:16:07.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12752025-09-03T04:16:07.492ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12762025-09-03T04:16:07.497ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
12772025-09-03T04:16:07.497ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
12782025-09-03T04:16:07.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12792025-09-03T04:16:07.498ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
12802025-09-03T04:16:07.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
12812025-09-03T04:16:07.500ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
12822025-09-03T04:16:07.500ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 145.238482ms iteration = 25 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
12832025-09-03T04:16:07.500ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = cf400120-81cc-41d9-8da6-7e88a5d7c686 dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
12842025-09-03T04:16:07.500ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
12852025-09-03T04:16:07.500ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-09-03 04:04:24.694355 UTC dns_group = internal generation = 1
12862025-09-03T04:16:07.500ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 147.82081ms iteration = 13 status = {"generation":1}
12872025-09-03T04:16:07.502ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12882025-09-03T04:16:07.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12892025-09-03T04:16:07.505ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12902025-09-03T04:16:07.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12912025-09-03T04:16:07.507ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
12922025-09-03T04:16:07.508ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 60e73e54-9659-486a-96ba-b69df28d8796 original_time_created = 2025-09-03 04:03:56.692551 UTC target_id = 60e73e54-9659-486a-96ba-b69df28d8796 time_created = 2025-09-03 04:03:56.692551 UTC
12932025-09-03T04:16:07.508ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12942025-09-03T04:16:07.508ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 153.394961ms iteration = 73 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"60e73e54-9659-486a-96ba-b69df28d8796","time_created":"2025-09-03T04:03:56.692551Z"}
12952025-09-03T04:16:07.510ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
12962025-09-03T04:16:07.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
12972025-09-03T04:16:07.512ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12982025-09-03T04:16:07.512ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 6ba5229c-c111-45f6-88aa-7f00b514c681, lookup_type: ById(6ba5229c-c111-45f6-88aa-7f00b514c681) } result = Ok(())
12992025-09-03T04:16:07.512ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13002025-09-03T04:16:07.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13012025-09-03T04:16:07.514ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 160.18498ms iteration = 25 status = 0
13022025-09-03T04:16:07.516ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
13032025-09-03T04:16:07.516ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13042025-09-03T04:16:07.516ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-09-03T04:04:05.025624Z, time_modified: 2025-09-03T04:04:05.025624Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1, db_silo: Silo { identity: SiloIdentity { id: 1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-09-03T04:04:25.421556Z, time_modified: 2025-09-03T04:04:25.421556Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "88177654dc1555d3c82c6767dd227356fa644067c4cbd354dce9bb9fa68212ac" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1, db_silo: Silo { identity: SiloIdentity { id: 1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-09-03T04:04:25.421556Z, time_modified: 2025-09-03T04:04:25.421556Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "88177654dc1555d3c82c6767dd227356fa644067c4cbd354dce9bb9fa68212ac" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:94
13052025-09-03T04:16:07.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
13062025-09-03T04:16:07.516ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 163.982657ms iteration = 13 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1","tls_certs":["88177654dc1555d3c82c6767dd227356fa644067c4cbd354dce9bb9fa68212ac"]}},"default_endpoint":{"silo_id":"1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1","tls_certs":["88177654dc1555d3c82c6767dd227356fa644067c4cbd354dce9bb9fa68212ac"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
13072025-09-03T04:16:07.518ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13082025-09-03T04:16:07.521ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
13092025-09-03T04:16:07.521ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13102025-09-03T04:16:07.521ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
13112025-09-03T04:16:07.521ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:138 found = 0 not_ready_to_be_deleted = 0
13122025-09-03T04:16:07.521ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 166.449345ms iteration = 13 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
13132025-09-03T04:16:07.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
13142025-09-03T04:16:07.531ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 176.325347ms iteration = 73 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
13152025-09-03T04:16:07.554ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1214 zone_address = fd00:1122:3344:101::2
13162025-09-03T04:16:07.554ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
13172025-09-03T04:16:07.554ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f39025b6-bbde-4371-86c5-36c46eb2c56d", "content-length": "26", "date": "Wed, 03 Sep 2025 04:16:07 GMT"} })
13182025-09-03T04:16:07.554ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1217 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
13192025-09-03T04:16:07.556ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1251 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
13202025-09-03T04:16:07.586ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
13212025-09-03T04:16:07.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
13222025-09-03T04:16:07.589ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
13232025-09-03T04:16:07.589ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 235.259303ms iteration = 13 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
13242025-09-03T04:16:07.639ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
13252025-09-03T04:16:07.662ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1214 zone_address = fd00:1122:3344:101::2
13262025-09-03T04:16:07.662ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
13272025-09-03T04:16:07.662ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "02f66115-85f0-4087-a8c0-504aa5f4ebff", "content-length": "26", "date": "Wed, 03 Sep 2025 04:16:07 GMT"} })
13282025-09-03T04:16:07.662ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1217 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
13292025-09-03T04:16:07.665ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1251 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
13302025-09-03T04:16:07.705ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1214 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 zone_address = fd00:1122:3344:101::2
13312025-09-03T04:16:07.705ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
13322025-09-03T04:16:07.705ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a5a1928-b7ee-49fa-b6b5-7173078bf83c", "content-length": "26", "date": "Wed, 03 Sep 2025 04:16:07 GMT"} })
13332025-09-03T04:16:07.705ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1217 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
13342025-09-03T04:16:07.707ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1251 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40
13352025-09-03T04:16:07.715ZINFOnexus (BackgroundTasks): VPC route manager sled 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
13362025-09-03T04:16:07.715ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
13372025-09-03T04:16:07.716ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "909f0d60-874a-433e-892a-3664e04f4099", "content-length": "372", "date": "Wed, 03 Sep 2025 04:16:07 GMT"} })
13382025-09-03T04:16:07.716ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13392025-09-03T04:16:07.716ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13402025-09-03T04:16:07.734ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
13412025-09-03T04:16:07.734ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
13422025-09-03T04:16:07.735ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "418a9a67-7c1c-490e-8e45-73c995569acc", "date": "Wed, 03 Sep 2025 04:16:07 GMT"} })
13432025-09-03T04:16:07.735ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13442025-09-03T04:16:07.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13452025-09-03T04:16:07.738ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13462025-09-03T04:16:07.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13472025-09-03T04:16:07.741ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13482025-09-03T04:16:07.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13492025-09-03T04:16:07.745ZINFOnexus (BackgroundTasks): VPC route manager sled 6ba5229c-c111-45f6-88aa-7f00b514c681 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
13502025-09-03T04:16:07.745ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
13512025-09-03T04:16:07.745ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "613548fb-d4a9-4518-a8e7-91a3b98af73f", "date": "Wed, 03 Sep 2025 04:16:07 GMT"} })
13522025-09-03T04:16:07.747ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 393.148746ms iteration = 25 status = {}
13532025-09-03T04:16:07.859ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 506.664178ms iteration = 25 status = {}
13542025-09-03T04:16:07.869ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
13552025-09-03T04:16:07.870ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "774ef256-c8ce-4610-a06a-36ac89bfa228", "date": "Wed, 03 Sep 2025 04:16:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13562025-09-03T04:16:07.870ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13572025-09-03T04:16:07.870ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
13582025-09-03T04:16:07.871ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "e225e703-9563-4f3c-8b25-764d9e9f53d8", "content-length": "691", "date": "Wed, 03 Sep 2025 04:16:07 GMT"} })
13592025-09-03T04:16:07.871ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
13602025-09-03T04:16:07.871ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
13612025-09-03T04:16:07.871ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
13622025-09-03T04:16:07.872ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "66956cb0-6c90-4acc-8b17-e0aba4f7cdc2", "date": "Wed, 03 Sep 2025 04:16:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13632025-09-03T04:16:07.872ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13642025-09-03T04:16:07.874ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 519.3893ms iteration = 25 status = {}
13652025-09-03T04:16:07.875ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13662025-09-03T04:16:07.875ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 522.991076ms iteration = 15 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
13672025-09-03T04:16:07.878ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
13682025-09-03T04:16:07.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
13692025-09-03T04:16:07.881ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 528.516666ms iteration = 25 status = null
13702025-09-03T04:16:07.898ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
13712025-09-03T04:16:07.960ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
13722025-09-03T04:16:07.961ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "fda5f14f-05f8-4e8d-aa58-aa560b7c3bfb", "date": "Wed, 03 Sep 2025 04:16:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
13732025-09-03T04:16:07.961ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
13742025-09-03T04:16:07.963ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "7d07129e-3f83-4885-95e3-e3ed212a5719", "date": "Wed, 03 Sep 2025 04:16:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
13752025-09-03T04:16:07.963ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
13762025-09-03T04:16:07.965ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "563b0564-a36b-4915-9cae-fe7d72242176", "date": "Wed, 03 Sep 2025 04:16:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
13772025-09-03T04:16:07.965ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
13782025-09-03T04:16:07.967ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
13792025-09-03T04:16:07.967ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 615.329374ms iteration = 15 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
13802025-09-03T04:16:07.980ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13812025-09-03T04:16:07.980ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13822025-09-03T04:16:07.983ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 6d67bb72-7d81-4e35-9440-7d645a356abd, rack_id: b1c08780-6eb9-4097-acea-f38d0539ff40, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(2c3aab11-bea9-412a-a4d5-9f98fa9ac796) } rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch_location = Switch0
13832025-09-03T04:16:07.983ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13842025-09-03T04:16:07.983ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13852025-09-03T04:16:07.995ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
13862025-09-03T04:16:07.995ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 2c3aab11-bea9-412a-a4d5-9f98fa9ac796
13872025-09-03T04:16:07.995ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
13882025-09-03T04:16:07.995ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
13892025-09-03T04:16:07.995ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
13902025-09-03T04:16:07.995ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
13912025-09-03T04:16:07.995ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
13922025-09-03T04:16:07.995ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
13932025-09-03T04:16:07.995ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
13942025-09-03T04:16:07.995ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
13952025-09-03T04:16:07.996ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40
13962025-09-03T04:16:07.996ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13972025-09-03T04:16:07.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13982025-09-03T04:16:07.999ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch_location = Switch0
13992025-09-03T04:16:07.999ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
14002025-09-03T04:16:08.099ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "0323babe-4c15-4305-9d8c-561b2bcac4a8", "date": "Wed, 03 Sep 2025 04:16:07 GMT"} })
14012025-09-03T04:16:08.099ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch = Switch0
14022025-09-03T04:16:08.099ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
14032025-09-03T04:16:08.099ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "1f94b870-b0e3-423e-8bea-171d4e8ec90f", "content-length": "682", "date": "Wed, 03 Sep 2025 04:16:07 GMT"} })
14042025-09-03T04:16:08.099ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14052025-09-03T04:16:08.100ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14062025-09-03T04:16:08.103ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14072025-09-03T04:16:08.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14082025-09-03T04:16:08.105ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14092025-09-03T04:16:08.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14102025-09-03T04:16:08.108ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14112025-09-03T04:16:08.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14122025-09-03T04:16:08.115ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 761.06627ms iteration = 25 status = {}
14132025-09-03T04:16:09.654ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 387 local_addr = [fd00:1122:3344:101::a]:33204 method = GET remote_addr = [fd00:1122:3344:101::d]:55526 req_id = b703a5b4-26b3-425c-809c-dc2f9a947918 response_code = 200 uri = /fa7984b8-4f16-449d-b4e0-b61c280b0e91
14142025-09-03T04:16:12.354ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 146 reason = Timeout
14152025-09-03T04:16:12.354ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
14162025-09-03T04:16:12.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
14172025-09-03T04:16:12.356ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14182025-09-03T04:16:12.360ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
14192025-09-03T04:16:12.360ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
14202025-09-03T04:16:12.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
14212025-09-03T04:16:12.361ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
14222025-09-03T04:16:12.361ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.958441ms iteration = 146 status = {"chicken_switches_updated":false}
14232025-09-03T04:16:17.355ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 74 reason = Timeout
14242025-09-03T04:16:17.355ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 74 reason = Timeout
14252025-09-03T04:16:17.355ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14262025-09-03T04:16:17.355ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 147 reason = Timeout
14272025-09-03T04:16:17.355ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
14282025-09-03T04:16:17.355ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
14292025-09-03T04:16:17.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
14302025-09-03T04:16:17.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
14312025-09-03T04:16:17.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
14322025-09-03T04:16:17.356ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14332025-09-03T04:16:17.356ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14342025-09-03T04:16:17.357ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
14352025-09-03T04:16:17.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
14362025-09-03T04:16:17.358ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.636896ms iteration = 74 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
14372025-09-03T04:16:17.360ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14382025-09-03T04:16:17.360ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14392025-09-03T04:16:17.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
14402025-09-03T04:16:17.360ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
14412025-09-03T04:16:17.360ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
14422025-09-03T04:16:17.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
14432025-09-03T04:16:17.361ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14442025-09-03T04:16:17.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
14452025-09-03T04:16:17.362ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
14462025-09-03T04:16:17.362ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.057151ms iteration = 147 status = {"chicken_switches_updated":false}
14472025-09-03T04:16:17.362ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14482025-09-03T04:16:17.365ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 60e73e54-9659-486a-96ba-b69df28d8796, lookup_type: ById(60e73e54-9659-486a-96ba-b69df28d8796) } result = Ok(())
14492025-09-03T04:16:17.365ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14502025-09-03T04:16:17.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
14512025-09-03T04:16:17.381ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 60e73e54-9659-486a-96ba-b69df28d8796 original_time_created = 2025-09-03 04:03:56.692551 UTC target_id = 60e73e54-9659-486a-96ba-b69df28d8796 time_created = 2025-09-03 04:03:56.692551 UTC
14522025-09-03T04:16:17.381ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.20774ms iteration = 74 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"60e73e54-9659-486a-96ba-b69df28d8796","time_created":"2025-09-03T04:03:56.692551Z"}
14532025-09-03T04:16:19.654ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 361 local_addr = [fd00:1122:3344:101::a]:33204 method = GET remote_addr = [fd00:1122:3344:101::d]:55526 req_id = 30f0a2b5-db19-49d5-b04e-dd4f7a034e8f response_code = 200 uri = /fa7984b8-4f16-449d-b4e0-b61c280b0e91
14542025-09-03T04:16:22.355ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 148 reason = Timeout
14552025-09-03T04:16:22.355ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
14562025-09-03T04:16:22.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
14572025-09-03T04:16:22.357ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14582025-09-03T04:16:22.360ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
14592025-09-03T04:16:22.360ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
14602025-09-03T04:16:22.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
14612025-09-03T04:16:22.362ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
14622025-09-03T04:16:22.362ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.044002ms iteration = 148 status = {"chicken_switches_updated":false}
14632025-09-03T04:16:27.355ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 75 reason = Timeout
14642025-09-03T04:16:27.355ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
14652025-09-03T04:16:27.355ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 75 reason = Timeout
14662025-09-03T04:16:27.355ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 149 reason = Timeout
14672025-09-03T04:16:27.355ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14682025-09-03T04:16:27.355ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
14692025-09-03T04:16:27.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
14702025-09-03T04:16:27.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
14712025-09-03T04:16:27.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
14722025-09-03T04:16:27.357ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14732025-09-03T04:16:27.357ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
14742025-09-03T04:16:27.357ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14752025-09-03T04:16:27.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
14762025-09-03T04:16:27.359ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.463345ms iteration = 75 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
14772025-09-03T04:16:27.360ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14782025-09-03T04:16:27.360ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14792025-09-03T04:16:27.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
14802025-09-03T04:16:27.361ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
14812025-09-03T04:16:27.361ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
14822025-09-03T04:16:27.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
14832025-09-03T04:16:27.362ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14842025-09-03T04:16:27.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
14852025-09-03T04:16:27.363ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
14862025-09-03T04:16:27.363ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.573685ms iteration = 149 status = {"chicken_switches_updated":false}
14872025-09-03T04:16:27.363ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14882025-09-03T04:16:27.366ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 60e73e54-9659-486a-96ba-b69df28d8796, lookup_type: ById(60e73e54-9659-486a-96ba-b69df28d8796) } result = Ok(())
14892025-09-03T04:16:27.366ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14902025-09-03T04:16:27.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
14912025-09-03T04:16:27.378ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 60e73e54-9659-486a-96ba-b69df28d8796 original_time_created = 2025-09-03 04:03:56.692551 UTC target_id = 60e73e54-9659-486a-96ba-b69df28d8796 time_created = 2025-09-03 04:03:56.692551 UTC
14922025-09-03T04:16:27.378ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.073877ms iteration = 75 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"60e73e54-9659-486a-96ba-b69df28d8796","time_created":"2025-09-03T04:03:56.692551Z"}
14932025-09-03T04:16:29.654ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 565 local_addr = [fd00:1122:3344:101::a]:33204 method = GET remote_addr = [fd00:1122:3344:101::d]:55526 req_id = b8d9e6f9-4325-42fa-a300-f051daba7d74 response_code = 200 uri = /fa7984b8-4f16-449d-b4e0-b61c280b0e91
14942025-09-03T04:16:32.355ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 150 reason = Timeout
14952025-09-03T04:16:32.355ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
14962025-09-03T04:16:32.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
14972025-09-03T04:16:32.358ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14982025-09-03T04:16:32.360ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
14992025-09-03T04:16:32.360ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
15002025-09-03T04:16:32.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
15012025-09-03T04:16:32.362ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
15022025-09-03T04:16:32.362ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.822799ms iteration = 150 status = {"chicken_switches_updated":false}
15032025-09-03T04:16:37.352ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 26 reason = Timeout
15042025-09-03T04:16:37.352ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 26 reason = Timeout
15052025-09-03T04:16:37.352ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 26 reason = Timeout
15062025-09-03T04:16:37.352ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
15072025-09-03T04:16:37.352ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
15082025-09-03T04:16:37.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
15092025-09-03T04:16:37.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
15102025-09-03T04:16:37.354ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 26 reason = Timeout
15112025-09-03T04:16:37.354ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 26 reason = Timeout
15122025-09-03T04:16:37.354ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 26 reason = Timeout
15132025-09-03T04:16:37.354ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
15142025-09-03T04:16:37.354ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 20.34\u{b5}s iteration = 26 status = {"disabled":true,"errors":[],"sps":[]}
15152025-09-03T04:16:37.354ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
15162025-09-03T04:16:37.354ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 26 reason = Timeout
15172025-09-03T04:16:37.354ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15182025-09-03T04:16:37.354ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 76 reason = Timeout
15192025-09-03T04:16:37.354ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 26 reason = Timeout
15202025-09-03T04:16:37.354ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 26 reason = Timeout
15212025-09-03T04:16:37.354ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
15222025-09-03T04:16:37.354ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15232025-09-03T04:16:37.354ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15242025-09-03T04:16:37.354ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 76 reason = Timeout
15252025-09-03T04:16:37.354ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
15262025-09-03T04:16:37.354ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 26 reason = Timeout
15272025-09-03T04:16:37.354ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
15282025-09-03T04:16:37.354ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 151 reason = Timeout
15292025-09-03T04:16:37.354ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15302025-09-03T04:16:37.354ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
15312025-09-03T04:16:37.354ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 26 reason = Timeout
15322025-09-03T04:16:37.354ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 26 reason = Timeout
15332025-09-03T04:16:37.354ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
15342025-09-03T04:16:37.354ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 26 reason = Timeout
15352025-09-03T04:16:37.354ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
15362025-09-03T04:16:37.354ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
15372025-09-03T04:16:37.354ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 26 reason = Timeout
15382025-09-03T04:16:37.354ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
15392025-09-03T04:16:37.354ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 26 reason = Timeout
15402025-09-03T04:16:37.354ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
15412025-09-03T04:16:37.354ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 28 reason = Timeout
15422025-09-03T04:16:37.355ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15432025-09-03T04:16:37.355ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 26 reason = Timeout
15442025-09-03T04:16:37.355ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
15452025-09-03T04:16:37.355ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 26 reason = Timeout
15462025-09-03T04:16:37.355ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
15472025-09-03T04:16:37.355ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
15482025-09-03T04:16:37.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
15492025-09-03T04:16:37.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
15502025-09-03T04:16:37.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15512025-09-03T04:16:37.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
15522025-09-03T04:16:37.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
15532025-09-03T04:16:37.355ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1809, tv_nsec: 26863029 } })
15542025-09-03T04:16:37.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
15552025-09-03T04:16:37.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
15562025-09-03T04:16:37.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
15572025-09-03T04:16:37.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15582025-09-03T04:16:37.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
15592025-09-03T04:16:37.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
15602025-09-03T04:16:37.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
15612025-09-03T04:16:37.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
15622025-09-03T04:16:37.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
15632025-09-03T04:16:37.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
15642025-09-03T04:16:37.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
15652025-09-03T04:16:37.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15662025-09-03T04:16:37.355ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1206
15672025-09-03T04:16:37.356ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1809, tv_nsec: 28000597 } })
15682025-09-03T04:16:37.357ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1206
15692025-09-03T04:16:37.357ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
15702025-09-03T04:16:37.357ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 3.114183ms iteration = 26 status = {"errors":[],"finish_invoked_ok":[]}
15712025-09-03T04:16:37.358ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
15722025-09-03T04:16:37.358ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 6.147104ms iteration = 26 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
15732025-09-03T04:16:37.358ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15742025-09-03T04:16:37.358ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 3.503616ms iteration = 26 status = {"errors":[],"garbage_collect_requested":[]}
15752025-09-03T04:16:37.358ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15762025-09-03T04:16:37.358ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15772025-09-03T04:16:37.358ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15782025-09-03T04:16:37.358ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15792025-09-03T04:16:37.358ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15802025-09-03T04:16:37.358ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15812025-09-03T04:16:37.359ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.680774ms iteration = 76 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
15822025-09-03T04:16:37.359ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
15832025-09-03T04:16:37.359ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
15842025-09-03T04:16:37.359ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
15852025-09-03T04:16:37.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
15862025-09-03T04:16:37.362ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
15872025-09-03T04:16:37.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
15882025-09-03T04:16:37.363ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
15892025-09-03T04:16:37.363ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
15902025-09-03T04:16:37.363ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
15912025-09-03T04:16:37.363ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
15922025-09-03T04:16:37.364ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
15932025-09-03T04:16:37.364ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15942025-09-03T04:16:37.364ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
15952025-09-03T04:16:37.364ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15962025-09-03T04:16:37.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15972025-09-03T04:16:37.364ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
15982025-09-03T04:16:37.364ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15992025-09-03T04:16:37.364ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
16002025-09-03T04:16:37.364ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
16012025-09-03T04:16:37.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16022025-09-03T04:16:37.365ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 10.164824ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
16032025-09-03T04:16:37.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16042025-09-03T04:16:37.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
16052025-09-03T04:16:37.365ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
16062025-09-03T04:16:37.365ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
16072025-09-03T04:16:37.365ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
16082025-09-03T04:16:37.365ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16092025-09-03T04:16:37.365ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
16102025-09-03T04:16:37.365ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16112025-09-03T04:16:37.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
16122025-09-03T04:16:37.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16132025-09-03T04:16:37.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
16142025-09-03T04:16:37.366ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16152025-09-03T04:16:37.367ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
16162025-09-03T04:16:37.367ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
16172025-09-03T04:16:37.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
16182025-09-03T04:16:37.367ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1809, tv_nsec: 39225308 } })
16192025-09-03T04:16:37.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16202025-09-03T04:16:37.368ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
16212025-09-03T04:16:37.368ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 13.488628ms iteration = 151 status = {"chicken_switches_updated":false}
16222025-09-03T04:16:37.368ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1206 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40
16232025-09-03T04:16:37.368ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16242025-09-03T04:16:37.368ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16252025-09-03T04:16:37.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16262025-09-03T04:16:37.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
16272025-09-03T04:16:37.369ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16282025-09-03T04:16:37.369ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
16292025-09-03T04:16:37.369ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16302025-09-03T04:16:37.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16312025-09-03T04:16:37.371ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16322025-09-03T04:16:37.372ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
16332025-09-03T04:16:37.372ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
16342025-09-03T04:16:37.372ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
16352025-09-03T04:16:37.375ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 60e73e54-9659-486a-96ba-b69df28d8796, lookup_type: ById(60e73e54-9659-486a-96ba-b69df28d8796) } result = Ok(())
16362025-09-03T04:16:37.375ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16372025-09-03T04:16:37.377ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
16382025-09-03T04:16:37.403ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 60e73e54-9659-486a-96ba-b69df28d8796 original_time_created = 2025-09-03 04:03:56.692551 UTC target_id = 60e73e54-9659-486a-96ba-b69df28d8796 time_created = 2025-09-03 04:03:56.692551 UTC
16392025-09-03T04:16:37.403ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 48.893904ms iteration = 76 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"60e73e54-9659-486a-96ba-b69df28d8796","time_created":"2025-09-03T04:03:56.692551Z"}
16402025-09-03T04:16:37.430ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
16412025-09-03T04:16:37.430ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
16422025-09-03T04:16:37.431ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16432025-09-03T04:16:37.433ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
16442025-09-03T04:16:37.433ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 78.255307ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
16452025-09-03T04:16:37.433ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
16462025-09-03T04:16:37.433ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
16472025-09-03T04:16:37.433ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
16482025-09-03T04:16:37.434ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
16492025-09-03T04:16:37.435ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
16502025-09-03T04:16:37.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
16512025-09-03T04:16:37.440ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
16522025-09-03T04:16:37.441ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
16532025-09-03T04:16:37.441ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 86.072083ms iteration = 26 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
16542025-09-03T04:16:37.441ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 86.362505ms iteration = 26 status = {"errors":[],"requests_created_ok":[]}
16552025-09-03T04:16:37.441ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16562025-09-03T04:16:37.441ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16572025-09-03T04:16:37.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16582025-09-03T04:16:37.442ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
16592025-09-03T04:16:37.442ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 88.015697ms iteration = 26 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
16602025-09-03T04:16:37.443ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16612025-09-03T04:16:37.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16622025-09-03T04:16:37.444ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
16632025-09-03T04:16:37.444ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
16642025-09-03T04:16:37.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16652025-09-03T04:16:37.446ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16662025-09-03T04:16:37.447ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1214 zone_address = fd00:1122:3344:101::2
16672025-09-03T04:16:37.447ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
16682025-09-03T04:16:37.447ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d2bcb4cf-98bb-4c47-8461-a66d8045d40f", "content-length": "26", "date": "Wed, 03 Sep 2025 04:16:37 GMT"} })
16692025-09-03T04:16:37.447ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1217 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
16702025-09-03T04:16:37.449ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1251 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
16712025-09-03T04:16:37.450ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 6ba5229c-c111-45f6-88aa-7f00b514c681, lookup_type: ById(6ba5229c-c111-45f6-88aa-7f00b514c681) } result = Ok(())
16722025-09-03T04:16:37.450ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16732025-09-03T04:16:37.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16742025-09-03T04:16:37.455ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 100.992871ms iteration = 26 status = 0
16752025-09-03T04:16:37.460ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1214 zone_address = fd00:1122:3344:101::2
16762025-09-03T04:16:37.460ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
16772025-09-03T04:16:37.461ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "78f710d2-7e41-4c1c-b319-8a6b989160e7", "content-length": "26", "date": "Wed, 03 Sep 2025 04:16:37 GMT"} })
16782025-09-03T04:16:37.461ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1217 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
16792025-09-03T04:16:37.462ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1251 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
16802025-09-03T04:16:37.473ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1214 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 zone_address = fd00:1122:3344:101::2
16812025-09-03T04:16:37.473ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
16822025-09-03T04:16:37.473ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b1586fc-45d6-4bfa-9fe5-147a4fb34b5d", "content-length": "26", "date": "Wed, 03 Sep 2025 04:16:37 GMT"} })
16832025-09-03T04:16:37.473ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1217 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
16842025-09-03T04:16:37.475ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1251 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40
16852025-09-03T04:16:37.488ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 133.99446ms iteration = 26 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
16862025-09-03T04:16:37.489ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 134.934077ms iteration = 28 status = {"physical_disks_added":0}
16872025-09-03T04:16:37.493ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
16882025-09-03T04:16:37.493ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 138.339162ms iteration = 26 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
16892025-09-03T04:16:37.528ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
16902025-09-03T04:16:37.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
16912025-09-03T04:16:37.529ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16922025-09-03T04:16:37.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
16932025-09-03T04:16:37.531ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 179.080207ms iteration = 26 status = null
16942025-09-03T04:16:37.537ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16952025-09-03T04:16:37.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
16962025-09-03T04:16:37.539ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16972025-09-03T04:16:37.540ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 188.179743ms iteration = 26 status = {}
16982025-09-03T04:16:37.541ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
16992025-09-03T04:16:37.541ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
17002025-09-03T04:16:37.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
17012025-09-03T04:16:37.543ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17022025-09-03T04:16:37.545ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
17032025-09-03T04:16:37.545ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
17042025-09-03T04:16:37.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
17052025-09-03T04:16:37.554ZINFOnexus (BackgroundTasks): VPC route manager sled 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
17062025-09-03T04:16:37.554ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
17072025-09-03T04:16:37.555ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3c0a895-fdfe-4b89-9a86-1e8732495f69", "content-length": "372", "date": "Wed, 03 Sep 2025 04:16:37 GMT"} })
17082025-09-03T04:16:37.555ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17092025-09-03T04:16:37.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17102025-09-03T04:16:37.568ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
17112025-09-03T04:16:37.568ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
17122025-09-03T04:16:37.568ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "d879606d-3387-42f6-9b33-b3d419ab8310", "date": "Wed, 03 Sep 2025 04:16:37 GMT"} })
17132025-09-03T04:16:37.568ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17142025-09-03T04:16:37.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17152025-09-03T04:16:37.571ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17162025-09-03T04:16:37.571ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17172025-09-03T04:16:37.574ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17182025-09-03T04:16:37.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17192025-09-03T04:16:37.577ZINFOnexus (BackgroundTasks): VPC route manager sled 6ba5229c-c111-45f6-88aa-7f00b514c681 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
17202025-09-03T04:16:37.577ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
17212025-09-03T04:16:37.577ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "d8001512-f382-48bc-984e-2c3aca77748f", "date": "Wed, 03 Sep 2025 04:16:37 GMT"} })
17222025-09-03T04:16:37.579ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 224.234974ms iteration = 26 status = {}
17232025-09-03T04:16:37.587ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
17242025-09-03T04:16:37.587ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "f4bf1b4c-6afb-47e4-bcd2-a44118c953d5", "content-length": "691", "date": "Wed, 03 Sep 2025 04:16:37 GMT"} })
17252025-09-03T04:16:37.587ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
17262025-09-03T04:16:37.587ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
17272025-09-03T04:16:37.587ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
17282025-09-03T04:16:37.588ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 233.35176ms iteration = 26 status = {}
17292025-09-03T04:16:37.619ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17302025-09-03T04:16:37.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17312025-09-03T04:16:37.621ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 6d67bb72-7d81-4e35-9440-7d645a356abd, rack_id: b1c08780-6eb9-4097-acea-f38d0539ff40, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(2c3aab11-bea9-412a-a4d5-9f98fa9ac796) } rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch_location = Switch0
17322025-09-03T04:16:37.621ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17332025-09-03T04:16:37.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17342025-09-03T04:16:37.630ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
17352025-09-03T04:16:37.630ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 2c3aab11-bea9-412a-a4d5-9f98fa9ac796
17362025-09-03T04:16:37.631ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
17372025-09-03T04:16:37.631ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
17382025-09-03T04:16:37.631ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
17392025-09-03T04:16:37.631ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
17402025-09-03T04:16:37.631ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
17412025-09-03T04:16:37.631ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
17422025-09-03T04:16:37.631ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
17432025-09-03T04:16:37.631ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
17442025-09-03T04:16:37.631ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40
17452025-09-03T04:16:37.631ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17462025-09-03T04:16:37.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17472025-09-03T04:16:37.633ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch_location = Switch0
17482025-09-03T04:16:37.633ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
17492025-09-03T04:16:37.812ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "04d243b0-d6fb-4483-86af-fd47c18e3631", "date": "Wed, 03 Sep 2025 04:16:37 GMT"} })
17502025-09-03T04:16:37.812ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch = Switch0
17512025-09-03T04:16:37.812ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
17522025-09-03T04:16:37.817ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b3fccf1-811c-475e-80c9-136dd01a446a", "content-length": "682", "date": "Wed, 03 Sep 2025 04:16:37 GMT"} })
17532025-09-03T04:16:37.817ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17542025-09-03T04:16:37.818ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17552025-09-03T04:16:37.822ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17562025-09-03T04:16:37.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17572025-09-03T04:16:37.826ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17582025-09-03T04:16:37.826ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17592025-09-03T04:16:37.828ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17602025-09-03T04:16:37.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17612025-09-03T04:16:37.834ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 479.620721ms iteration = 26 status = {}
17622025-09-03T04:16:39.654ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 528 local_addr = [fd00:1122:3344:101::a]:33204 method = GET remote_addr = [fd00:1122:3344:101::d]:55526 req_id = 5acf7f29-9124-49a1-8bcd-511340ac5daf response_code = 200 uri = /fa7984b8-4f16-449d-b4e0-b61c280b0e91
17632025-09-03T04:16:42.355ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 152 reason = Timeout
17642025-09-03T04:16:42.355ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
17652025-09-03T04:16:42.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
17662025-09-03T04:16:42.357ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17672025-09-03T04:16:42.360ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
17682025-09-03T04:16:42.360ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
17692025-09-03T04:16:42.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
17702025-09-03T04:16:42.362ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
17712025-09-03T04:16:42.362ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.414307ms iteration = 152 status = {"chicken_switches_updated":false}
17722025-09-03T04:16:47.355ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 77 reason = Timeout
17732025-09-03T04:16:47.355ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
17742025-09-03T04:16:47.355ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 77 reason = Timeout
17752025-09-03T04:16:47.355ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 153 reason = Timeout
17762025-09-03T04:16:47.355ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
17772025-09-03T04:16:47.355ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17782025-09-03T04:16:47.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
17792025-09-03T04:16:47.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
17802025-09-03T04:16:47.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
17812025-09-03T04:16:47.357ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
17822025-09-03T04:16:47.357ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17832025-09-03T04:16:47.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
17842025-09-03T04:16:47.357ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17852025-09-03T04:16:47.358ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.448575ms iteration = 77 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
17862025-09-03T04:16:47.361ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
17872025-09-03T04:16:47.361ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17882025-09-03T04:16:47.361ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
17892025-09-03T04:16:47.361ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
17902025-09-03T04:16:47.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
17912025-09-03T04:16:47.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
17922025-09-03T04:16:47.362ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17932025-09-03T04:16:47.363ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
17942025-09-03T04:16:47.363ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
17952025-09-03T04:16:47.363ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.843727ms iteration = 153 status = {"chicken_switches_updated":false}
17962025-09-03T04:16:47.364ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17972025-09-03T04:16:47.367ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 60e73e54-9659-486a-96ba-b69df28d8796, lookup_type: ById(60e73e54-9659-486a-96ba-b69df28d8796) } result = Ok(())
17982025-09-03T04:16:47.367ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17992025-09-03T04:16:47.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
18002025-09-03T04:16:47.378ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 60e73e54-9659-486a-96ba-b69df28d8796 original_time_created = 2025-09-03 04:03:56.692551 UTC target_id = 60e73e54-9659-486a-96ba-b69df28d8796 time_created = 2025-09-03 04:03:56.692551 UTC
18012025-09-03T04:16:47.378ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.37058ms iteration = 77 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"60e73e54-9659-486a-96ba-b69df28d8796","time_created":"2025-09-03T04:03:56.692551Z"}